Systems and methods for collaborative synchronous image selection

ABSTRACT

Systems and methods for real-time collaborative computing and collective intelligence are disclosed. A collaborative application runs on a collaborative server connected to a plurality of computing devices. Collaborative sessions are run wherein a group of independent users, networked over the internet, collaboratively answer questions in real-time, thereby harnessing their collective intelligence. Methods for collaborative selection of a target point on an image displayed on the computing devices. Methods for collaborative zooming and panning of an image displayed on the computing devices.

This application claims the benefit of U.S. Provisional Application No.62/117,808 entitled SYSTEM AND METHODS FOR COLLABORATIVE SYNCHRONOUSIMAGE SELECTION, filed Feb. 18, 2015, which is incorporated in itsentirety herein by reference.

This application is a continuation-in-part of U.S. application Ser. No.14/668,970 entitled METHODS AND SYSTEMS FOR REAL-TIME CLOSED-LOOPCOLLABORATIVE INTELLIGENCE, filed Mar. 25, 2015, which in turns claimsthe benefit of U.S. Provisional Application 61/970,885 entitled METHODAND SYSTEM FOR ENABLING A GROUPWISE COLLABORATIVE CONSCIOUSNESS, filedMar. 26, 2014, both of which are incorporated in their entirety hereinby reference.

This application is a continuation-in-part of U.S. application Ser. No.14/708,038 entitled MULTI-GROUP METHODS AND SYSTEMS FOR REAL-TIMEMULTI-TIER COLLABORATIVE INTELLIGENCE, filed May 8, 2015, which in turnsclaims the benefit of U.S. Provisional Application 61/991,505 entitledMETHODS AND SYSTEM FOR MULTI-TIER COLLABORATIVE INTELLIGENCE, filed May10, 2014, both of which are incorporated in their entirety herein byreference.

This application is a continuation-in-part of U.S. application Ser. No.14/738,768 entitled INTUITIVE INTERFACES FOR REAL-TIME COLLABORATIVEINTELLIGENCE, filed Jun. 12, 2015, which in turns claims the benefit ofU.S. Provisional Application 62/012,403 entitled INTUITIVE INTERFACE FORREAL-TIME COLLABORATIVE CONTROL, filed Jun. 15, 2014, both of which areincorporated in their entirety herein by reference.

This application is a continuation-in-part of U.S. application Ser. No.14/859,035 entitled SYSTEMS AND METHODS FOR ASSESSMENT AND OPTIMIZATIONOF REAL-TIME COLLABORATIVE INTELLIGENCE SYSTEMS, filed Sep. 18, 2015which in turns claims the benefit of U.S. Provisional Application No.62/066,718 entitled SYSTEM AND METHOD FOR MODERATING AND OPTIMIZINGREAL-TIME SWARM INTELLIGENCES, filed Oct. 21, 2014, both of which areincorporated in their entirety herein by reference.

This application is a continuation-in-part of U.S. application Ser. No.14/920,819 entitled SUGGESTION AND BACKGROUND MODES FOR REAL-TIMECOLLABORATIVE INTELLIGENCE SYSTEMS, filed Oct. 22, 2015 which in turnsclaims the benefit of U.S. Provisional Application No. 62/067,505entitled SYSTEM AND METHODS FOR MODERATING REAL-TIME COLLABORATIVEDECISIONS OVER A DISTRIBUTED NETWORKS, filed Oct. 23, 2014.

This application is a continuation-in-part of U.S. application Ser. No.14/925,837 entitled MULTI-PHASE MULTI-GROUP SELECTION METHODS FORREAL-TIME COLLABORATIVE INTELLIGENCE SYSTEMS, filed Oct. 28, 2015 whichin turns claims the benefit of U.S. Provisional Application No.62/069,360 entitled SYSTEMS AND METHODS FOR ENABLING AND MODERATING AMASSIVELY-PARALLEL REAL-TIME SYNCHRONOUS COLLABORATIVESUPER-INTELLIGENCE, filed Oct. 28, 2014.

This application is a continuation-in-part of U.S. application Ser. No.15/017,424 entitled ITERATIVE SUGGESTION MODES FOR REAL-TIMECOLLABORATIVE INTELLIGENCE SYSTEMS, filed Feb. 5, 2016, which in turnsclaims the benefit of U.S. Provisional Application No. 62/113,393entitled SYSTEMS AND METHODS FOR ENABLING SYNCHRONOUS COLLABORATIVECREATIVITY AND DECISION MAKING, filed Feb. 7, 2015, both of which areincorporated in their entirety herein by reference.

This application is a continuation-in-part of International ApplicationNo. PCT/US15/22594, filed Mar. 25, 2015.

This application is a continuation-in-part of International ApplicationNo. PCT/US15/35694, filed Jun. 12, 2015.

This application is a continuation-in-part of International ApplicationNo. PCT/US15/56394, filed Oct. 20, 2015.

BACKGROUND OF THE INVENTION 1. Field of the Invention

The present invention relates generally to systems and methods forcollaborative intelligence, and more specifically to systems and methodsfor closed-loop, dynamic collaborative intelligence.

2. Discussion of the Related Art

Portable computing devices, such as cell phones, personal digitalassistants, and portable media players have become popular personaldevices due to their highly portable nature, their ability to provideaccessibility to a large library of stored media files, theirinterconnectivity with existing computer networks, and their ability topass information to other portable computing devices and/or tocentralized servers through phone networks, wireless networks and/orthrough local spontaneous networks such as Bluetooth® networks. Many ofthese devices also provide the ability to store and display media, suchas songs, videos, podcasts, ebooks, maps, and other related contentand/or programming. Many of these devices are also used as navigationtools, including GPS functionality. Many of these devices are also usedas personal communication devices, enabling phone, text, picture, andvideo communication with other similar portable devices. Many of thesedevices include touch screens, tilt interfaces, voice recognition, andother modern user input modes. As a result, the general social trendwithin industrial societies is that every person does now or soon willmaintain at least one such multi-purpose electronic device upon theirperson at most times, especially when out and about.

While such devices allow accessing information and person to personcommunication, they do not provide any unique tools and infrastructurethat specifically enable groups of electronically networked individualsto have a real-time group-wise experience that evokes the group'scollaborative intent and intelligence (Collaborative Consciousness).Hence, there is a substantial need to provide tools and methods by whichgroups of individuals, each having a portable computing device upontheir person, to more easily contribute their personal will/intent to anemerging collaborative consciousness, allowing the group to collectivelyanswer questions or otherwise express their groupwise will in real-time.Furthermore, there is a need to provide tools and methods that enablegroups of users to be informed of the group-wise will that is emergingin real-time. The present invention, as described herein, addressesthese and other deficiencies present in the art.

SUMMARY OF THE INVENTION

Several embodiments of the invention advantageously address the needsabove as well as other needs by providing an image selection process fora group of users in real-time collaborative control of at least onegraphical object, each user of the group associated with one of aplurality of computing devices, each computing device including adisplay interface, configured to exchange data with a collaborationserver, and run a collaboration application, the collaboration serverperforming the steps of: receiving, from a first computing device of thegroup, a question; sending of an image associated with the question tothe plurality of the computing devices, whereby at least a portion ofthe image is displayed on a target area of the display interface;moderating a collaborative selection process, whereby a target point onthe image is selected, wherein the target point has a location on theimage; and sending to the plurality of computing devices the location ofthe target point, whereby a graphical representation of the target pointlocation on the image is shown on the display interface.

In another embodiment, the invention can be characterized as a motionthreshold location selection process for a group of users in real-timecollaborative control of at least one graphical object, each user of thegroup associated with one of a plurality of computing devices, eachcomputing device including a display interface, configured to exchangedata with a collaboration server, and run a collaboration application,the collaboration server performing the steps of: determining if a speedof the graphical object has fallen below a motion threshold value; upondetermination that the speed of the graphical object has fallen belowthe motion threshold value, determining if the graphical object speedhas fallen below the motion threshold value for more than a timethreshold value; and upon determination that the graphical object speedhas fallen below the motion threshold value for more than the timethreshold value, selecting the current graphical object position as atarget position.

In yet another embodiment, the invention can be characterized as a forcethreshold location selection process for a group of users in real-timecollaborative control of at least one graphical object, each user of thegroup associated with one of a plurality of computing devices, eachcomputing device including a display interface, configured to exchangedata with a collaboration server, and run a collaboration application,the collaboration server performing the steps of: determining if amagnitude of a group intent vector has fallen below a force thresholdvalue; upon determination that the magnitude of the group intent vectorhas fallen below the force threshold value, determining if the magnitudeof the group intent vector has fallen below the force threshold valuefor more than a time threshold value; and upon determination that themagnitude of the group intent vector has fallen below the forcethreshold value for more than the time threshold value, selecting thecurrent graphical object position as a target position.

In yet another embodiment, the invention can be characterized as animage zoom process for a group of users in real-time collaborativecontrol of at least one graphical object, each user of the groupassociated with one of a plurality of computing devices, each computingdevice including a display interface, configured to exchange data with acollaboration server, and run a collaboration application, thecollaboration server performing the steps of: receiving a zoomindication, for an image displayed on each of the display interfaces andhaving a first zoom, from at least one computing device; determiningfrom the at least one zoom indications, a zoom change wherein the zoomchange; and sending a zoom signal to the plurality of computing devicesindicating the zoom change, whereby each display interface changes thefirst zoom by the zoom change.

In yet another embodiment, the invention can be characterized as animage panning process for a group of users in real-time collaborativecontrol of at least one graphical object, each user of the groupassociated with one of a plurality of computing devices, each computingdevice including a display interface displaying a portion of an image,configured to exchange data with a collaboration server, and run acollaboration application, the collaboration server performing the stepsof: determining that the at least one graphical object undercollaborative control has interacted with a panning boundary displayedon the display interface; determining a panning indication based atleast one the interaction of the at least one graphical object with thepanning boundary; and sending to the plurality of computing devices apanning indication, whereby the image is shown panned on the displayinterface.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features and advantages of severalembodiments of the present invention will be more apparent from thefollowing more particular description thereof, presented in conjunctionwith the following drawings.

FIG. 1 is a schematic diagram of an exemplary real-time collaborativesystem.

FIG. 2 is an exemplary display interface of a computing device of thecollaborative system in accordance with one embodiment of the presentinvention.

FIG. 3 is an exemplary group display interface of the computing deviceof the collaborative system at a point in time during a collaborationsession.

FIG. 4 is an exemplary group display interface of the computing deviceof the collaborative system after the collaboration session has beensuccessfully completed.

FIG. 5 is a snapshot of an exemplary collaborative session replay video.

FIG. 6 is an exemplary display of a time history session replay.

FIG. 7 is a snapshot of an exemplary pointer motion representationsession replay video.

FIG. 8 is an exemplary image target area shown during a collaborativesession.

FIG. 9 is a flowchart diagram of a method for location selection bymotion threshold during a collaborative session.

FIG. 10 is a flowchart diagram of a method for location selection byforce threshold during a collaborative session.

FIG. 11 is an exemplary image target area shown prior to the start of acollaborative session.

FIG. 12 is an exemplary image target area shown during a collaborativesession.

FIG. 13 is an exemplary image target area shown after a target image hasbeen selected by the collaborative session.

FIG. 14 is exemplary image target area, including a graphical pin, shownafter a target point has been selected by a collaborative session.

FIG. 15 is an exemplary map image target area shown prior to the startof a collaborative session.

FIG. 16 is an exemplary map image target area shown after completion ofthe collaborative session.

FIG. 17 is an exemplary calendar image target area shown during acollaborative session.

FIG. 18 is a flowchart diagram for a method for collaborative zoomingduring a collaborative session.

FIG. 19 is a panning-enabled map target area sequence during acollaborative session.

FIG. 20 is an exemplary map target area including a circular boundaryduring a collaborative session.

Corresponding reference characters indicate corresponding componentsthroughout the several views of the drawings. Skilled artisans willappreciate that elements in the figures are illustrated for simplicityand clarity and have not necessarily been drawn to scale. For example,the dimensions of some of the elements in the figures may be exaggeratedrelative to other elements to help to improve understanding of variousembodiments of the present invention. Also, common but well-understoodelements that are useful or necessary in a commercially feasibleembodiment are often not depicted in order to facilitate a lessobstructed view of these various embodiments of the present invention.

DETAILED DESCRIPTION

The following description is not to be taken in a limiting sense, but ismade merely for the purpose of describing the general principles ofexemplary embodiments. The scope of the invention should be determinedwith reference to the claims.

Reference throughout this specification to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment of the present invention. Thus,appearances of the phrases “in one embodiment,” “in an embodiment,” andsimilar language throughout this specification may, but do notnecessarily, all refer to the same embodiment.

Furthermore, the described features, structures, or characteristics ofthe invention may be combined in any suitable manner in one or moreembodiments. In the following description, numerous specific details areprovided, such as examples of programming, software modules, userselections, network transactions, database queries, database structures,hardware modules, hardware circuits, hardware chips, etc., to provide athorough understanding of embodiments of the invention. One skilled inthe relevant art will recognize, however, that the invention can bepracticed without one or more of the specific details, or with othermethods, components, materials, and so forth. In other instances,well-known structures, materials, or operations are not shown ordescribed in detail to avoid obscuring aspects of the invention.

As referred to in this specification, “media items” refers to video,audio, streaming and any combination thereof. In addition, the audiosubsystem is envisioned to optionally include features such as graphicequalization, volume, balance, fading, base and treble controls,surround sound emulation, and noise reduction. One skilled in therelevant art will appreciate that the above cited list of file formatsis not intended to be all inclusive.

Real-time occurrences as referenced herein are those that aresubstantially current within the context of human perception andreaction.

Prior systems for supporting networked collaboration have beendeveloped, but are asynchronous in nature, letting users provideisolated input, sequentially, the results generated by simple votes,ratings, and/or polling. For example, the present inventor was awardedU.S. Pat. No. 8,745,104 entitled “Collaborative Rejection of Media forPhysical Establishments” that describes a novel system for allowinggroups of users to reject the media playing in a restaurant or otherpublic venue through sequential polling. While effective for its desiredpurpose, such systems do not foster a true collaborative intelligenceamong groups of users. One reason that traditional polls and surveysfail to facilitate collaborative intelligence is that there is noreal-time negotiation, users instead imparting “fire and forget” votesor ratings such that collaborators can't explore the options inreal-time and converge on common ground. What is needed is a real-timeclosed loop system and supporting methods that enables groups ofnetworked users to function is a singular synchronous entity—a “socialswarm” that can make decision, take actions, or answer questions.

For example, the asynchronous nature of polls and surveys, as used bytools such as Reddit®, Yelp®, and Digg® cause distorted results. This isbecause the first few individuals to give a vote or rating have adisproportionate impact on the final output. The process is called“herding” or “snowballing” and reflects the fact that voters inasynchronous rating systems are deeply influenced by the votes thatprecede their own. In a recent study performed by researchers at MIT andHebrew University of Jerusalem (Muchnik et al.; Science Vol. 341, Aug.9, 2013), it was found that a single up-vote inserted as the firstresponse on a forum like Reddit, can boost the likelihood of futurepositive ratings by an average of 32% and can distort the outcome by25%. The researchers warn that these herding effects are distorting ourperceived views on everything from product recommendations on Amazon toelectoral polling and market predictions.

To avoid the pitfalls of asynchronous polling and enable a truereal-time collaborative intelligence, the present inventor has advancedcomputer mediated collaboration, transforming the process from thecollection of isolated asynchronous input to more sophisticatedclosed-loop systems that engender real-time synchronous collaboration.What is further needed, however, are systems and methods that enablegroups to select an image or a portion of an image through real-timesynchronous collaboration. The present invention discloses such systemsand methods, including novel systems and methods for placing a graphicalpin (or other locative identifier) on an image in a location that isselected collaboratively by a real-time synchronous group of users. Insome such embodiments, the image on which a graphical pin is placed is amap, a calendar, or a group of visually separable sub-images (forexample images of a set of products, logos, foods, or faces). Thepresent invention discloses novel systems and methods for zooming animage collaboratively by a real-time synchronous group of users.

Also disclosed herein is a new type of swarm replay wherein a groupintent vector is shown as a directional arrow across a plurality oftime-steps, enabling enhanced visualization of the decision.

The inventive systems and methods that enable a real-time, synchronous,Collaborative Intelligence that can generate and select from among aplurality of suggestions through a computer-moderated multi-rounditerative process. The present invention builds upon methods and systemsdisclosed in prior co-pending applications by the present inventor andhereby incorporated by reference.

As described in related patent application Ser. Nos. 14/668,970,14/708,038, 14/738,768 and 14/859,035 by the present inventor, all ofwhich are incorporated by reference, an innovative real-time system hasbeen developed that enables synchronous groups of users to form acoherent swarm intelligence that can collaboratively control a graphicalelement such as a pointer (or “puck”) in order to collectively answerquestions, make statements, take actions, select functions, or otherwiserespond to prompts. Using the disclosed systems and methods, a “socialswarming” platform is enabled that allows users to join one of aplurality of hosted swarms, collaborate with that swarm, earn scoresand/or credits and/or rankings based on their performance with respectto others in the swarm, and browse the stored output from other swarms.In some embodiments, groups (i.e. swarms) can compete with other swarms,each of said swarms also earning group-wise scores, credits, and/orrankings with respect to other swarms.

The collaboration system has been developed that allows the group ofusers to collaboratively control the graphical pointer in order tocollaboratively answer questions or otherwise respond to prompts. In oneembodiment, each individual user (“participant”) engages the userinterface on a computing device, conveying his or her individualreal-time will in response to a prompt such as a textually displayed (oraudibly displayed) question as well as in response to real-time feedbackprovided to the user of the group's emerging real-time intent. Thiscloses the loop around each user, for he is conveying individual intentwhile also reacting to the group's emerging intent. Thus each user mustbe able to see not only the prompt that begins a session, but thereal-time group intent as it is forming. For example, if the intent isbeing conveyed as words, the user will see those words form, letter byletter. If the intent is being conveyed as a direction, the user seesthe direction form, degree by degree. If the intent is being conveyed asa choice among objects, the user sees the graphical pointer 210 getcloser and closer to a particular chosen object. Thus, the user isseeing the group's will emerge before his eyes, reacting to that will inreal-time, and thus contributing to it. This closes the loop, not justaround one user, but around all users who have a similar experience ontheir own individual computing device at substantially the same time.While the embodiments described generally refer to portable computingdevices, it will be understood that non-portable computing devices, suchas desktop computers, may also be used.

Using the disclosed systems and methods, a “social swarming” platform isenabled that allows users to join one of a plurality of hosted groups(also referred to as swarms), each group comprising a plurality ofusers. The user may collaborate with that group, earn scores and/orcredits and/or rankings based on his performance with respect to othersin the group, and browse the stored output from other groups. In someembodiments, groups can compete with other groups, each of said groupsalso earning group scores, credits, and/or rankings with respect toother groups.

As described in co-pending U.S. patent application Ser. No. 14/668,970,METHODS AND SYSTEMS FOR REAL-TIME CLOSED-LOOP COLLABORATIVEINTELLIGENCE, by the present inventor, which is hereby incorporated byreference, systems and methods have been developed that allows a groupof users to collaboratively control the motion of the graphical pointerthrough a unique group-wise control paradigm. In some embodiments, thecollaboratively controlled pointer is configured to allow a group ofusers to choose letters, words, and/or numbers in response to a promptposed to the group. This enables the formation a group response that'snot based on the will of any individual user, but rather based on thecollective will of the group. In some embodiments, the group responsecomprises a set of letters, numbers, and/or words that form a textualreply to a question or statement posed to the group. In this way, thesystem disclosed herein enables a group of people to verballycommunicate as a single collective intelligence. In some embodiments,the group response may comprise a task performed in response to aprompt, for example the collaborative control of the pointer to draw apicture as a group. We refer to such a collective intelligence herein asa “swarm A.I.” for it's an artificial intellect that emerges from thegroup as a result of unique computer-moderated methods employed herein.

As described in co-pending U.S. patent application Ser. No. 14/708,038entitled MULTI-GROUP METHODS AND SYSTEMS FOR REAL-TIME MULTI-TIERCOLLABORATIVE INTELLIGENCE by the current inventor, which is herebyincorporated by reference, additional systems and methods were disclosedthat encourage groups of real-time users who are collaborativelyanswering questions by controlling the pointer, to produce coherentresponses while discouraging incoherent responses. A number of methodswere disclosed therein, including (a) coherence Scoring, (b) coherencefeedback, and (c) tiered processing. These and other techniques greatlyenhance the effectiveness of group-wise collaborative control,incentivizing participants to convey their individual will in a morecollaborative and coherent manner. By promoting greater collaborationamong members of a real-time group, the result a more focused “swarmintelligence” that produces more coherent answers in shorter amounts oftime.

As described in co-pending Provisional patent application Ser. No.14/738,768, entitled INTUITIVE INTERFACES FOR REAL-TIME COLLABORATIVEINTELLIGENCE, by the current inventor, which is hereby incorporated byreference, systems and additional novel methods were disclosed thatprovide additional novel systems and methods for user interfacing thatmake group-wise collaborative control more intuitive for theparticipating users. More specifically, novel systems and methods weredisclosed that enable individual users to intuitively impart their ownpersonal intent upon the graphical pointer that's being collaborativelycontrolled through real-time synchronous input collected from aplurality of users. Some disclosed embodiments support users interfacingwith a mouse, touchpad, trackball, or touchscreen. Other embodimentssupport users interfacing by tilting a mobile computing device such as aphone or tablet. Some preferred methods employ a manipulatable graphicalmagnet such that each user positions the magnet so as to impart asimulated pull on the group-wise pointer. The sum (or average) of theplurality of user's individual pulls are then used to influence thegroup-wise motion of the pointer, thus providing a physically intuitivecontrol paradigm. This and other novel solutions disclosed therein,solved the unique problems posed by enabling real-time synchronouscollaborative control of the group-wise pointer, thus enabling moreeffective “swarm intelligence” systems.

Also disclosed were methods and systems to enable individual users toquickly and easily pose questions to a collaborating group. As describedin the co-pending applications, moderating software can be configured toorchestrate which users can ask questions to the group and when. Once aquestion is asked, it can be answered through collaborative controlmethods in which the pointer is enabled to select one of a predefinedset of input choices (also referred to as answer choices) displayed(such as YES, NO, MAYBE, PROBABLY, DOUBTFUL) and/or by selecting one ofa set of alphanumeric characters. While such methods are highlyeffective, additional methods were disclosed herein that also allowsindividuals to specify a set of custom choices (i.e. possible answers)when asking a question to the group. The custom choices are thendisplayed graphically to the group as spatially arranged elements. Theusers can then collaboratively select among the spatially arrangedelements using the group-wise pointer. This allows for “bounded-setquestions” to be asked in a fast and fun manner, making the overall userexperience far more enjoyable and efficient.

Also disclosed were innovative hosting methods, allowing large numbersof users to login and participate in collaborative experiences. Morespecifically, a novel paradigm was disclosed that allows groups of usersto enter one of a plurality of on online collaboration groups (referredto herein as swarms) and participate in question/answer sessions. Userscan choose among the plurality of swarms, some being public (for groupsof strangers), other being private (for groups of friends). The swarmscan be general or themed, such that themed swarms are for questions on aparticular topic. In some embodiments, swarms can compete with otherswarms, bringing the real-time synchronous methods disclosed herein to ahigher order, creating a group comprised of groups (i.e. a swarm ofswarms). Also disclosed were methods of giving swarms their own uniquevoice by enabling answers produced by synchronous groups of users to beTweeted® (via Twitter®). This technology is the first known deploymentof Tweets from a consciousness comprised not of a single individual, buta swarm of individuals acting as emergent artificial intelligence.

As described in co-pending U.S. patent application Ser. No. 14/859,035,entitled SYSTEMS AND METHODS FOR ASSESSMENT AND OPTIMIZATION OFREAL-TIME COLLABORATIVE INTELLIGENCE SYSTEMS by the present inventor,and hereby incorporated by reference, systems and additional novelmethods were disclosed that enhance collaborative group decisions andactions. As disclosed, each swarm can be configured to dynamicallyadjust its configuration by adjusting the relative weighting of theinput received from members of the swarm. More specifically, in someembodiments, algorithms are used to increase the impact (weighting) thatsome users have upon the collective pull of the pointer, whiledecreasing the impact (weighting that other users have upon thecollective pull of the pointer. More specifically, user synchrony values(also called user synchronicity values) are computed to reflect eachuser's contribution to (or resistance of) an emerging consensus in eachtrial. Users who show a time history of high “sync”, have theirweighting values increased by the underlying software, while users whoshow a time history of low “sync” have their weighting values decrease.In this way, the swarm intelligence is adapted over time by theunderlying algorithms disclosed herein, strengthening the connections(i.e. input) with respect to the more collaborative users, and weakeningthe connections with respect to the less collaborative users. In thisway, the collaborative swarm in controlled in a manner innovativelyreminiscent of an adaptable neural network that optimizes itsintelligence over time

Also disclosed in Ser. No. 14/859,035 are methods for enabling membersof a swarm to ask a question to the group and then collect suggestionspopulate the possible answers from other members of the swarm. The groupthen collectively selects upon the options submitted. This technique isreferred to herein as “suggestion mode” and can be viewed as an enhancedversion of the “custom mode” disclosed in co-pending U.S. patentapplication Ser. No. 14/738,768. In the custom mode, a user can ask aquestion and provide a set of answers that the swarm will choose from.In suggestion mode, the user asks a question by indicates that thecustom choices are to be collected from any members of the swarm whohave ideas for options (i.e. suggestions) to populate the choices. Whena suggestion mode question is asked by a member of the swarm, all othermembers of the swarm are alerted with a pop-up window that asks them forsuggestions. A “suggestion timer” is optionally employed, giving thegroup a fixed time-window to provide suggestions. For example 30seconds. The system may be configured such that if a minimum of 2suggestions are not contributed within the 30 second window, thequestion is terminated. After all, you need at least two choices or theswarm has nothing to do. Conversely, if a full set of suggestions isreceived before the 30 second time limit is up, the question immediatelybegins, the group moving the pointer to choose among the suggestionsthat populated the board. In the event that more suggestions are madethan fit on the target area, the first suggestions received from usersare used by the software. Conversely, users with higher scores, ratings,sync values, or credit values, may be assigned priority makingsuggestions.

This method is highly effective, for it allows a swarm to engage in atwo-step question and answer process. For example, a swarm mightcomprise of a small group of friends. One user asks the group aquestion, entering it into the question box—“What movie should we seetonight?” The user selects suggestion mode. Immediately, that questionappears on the screens of all the other users. Also appearing on theirscreens is a suggestion dialog box, asking them to provide suggestionsand giving a time limit (for example, 30 seconds, counting down with atimer). As users enter suggestions, they appear around the target board.In addition, a chat window indicates the name of the user who made eachsuggestion. When the time is up, or when a full set of suggestions arereceived, the software starts the answer phase of the session, givingthe users collaborative control over the puck. The users thencollaboratively move the puck (using the real-time methods disclosed) toone of the choices. That choice is then sent to all members.

Still, new methods and systems are needed to enhance synchronouscollaborative decision making and bolster group intelligence. Herein,systems and methods that enable groups to select an image or a portionof an image through real-time synchronous collaboration. This includesnovel systems and methods for placing a graphical pin (or other locativeidentifier) on an image in a location that is selected collaborativelyby a real-time synchronous group. In some such embodiments, the image onwhich a graphical pin is placed is a map, a calendar, or a group ofvisually separable sub-images (for example images of a set of products,logos, foods, or faces). The present invention discloses novel systemsand methods for zooming an image collaboratively by a real-timesynchronous group of users.

Referring first to FIG. 1, a schematic diagram of an exemplarycollaboration system 100 is shown. Shown are a Central CollaborationServer (CCS) 102, a plurality of portable computing devices 104, and aplurality of exchanges of data with the Central Collaboration Server106.

Embodiments of the plurality of portable computing devices 104 and theinteraction of the computing devices 104 with the system 100 arepreviously disclosed in the related patent applications.

As shown in FIG. 1, the system 100 comprises the Central CollaborationServer (CCS) 102 in communication with the plurality of computingdevices 104, each of said computing devices 104 running a CollaborativeIntent Application (“CIA”). The system 100 is designed to enable theplurality of users, each engaging an interface of one of said computingdevices 104, to jointly control a single graphical object, for example amovable graphical pointer 210, through real-time group-wisecollaboration. In some embodiments, such as a multi-tier architecture,the portable computing devices 104 may communicate with each other. TheCCS 102 includes software and additional elements as necessary toperform the required functions. In this application, it will beunderstood that the term “CCS” may be used to refer to the software ofthe CCS 102 or other elements of the CCS 102 that are performing thegiven function.

Although multiple pointers controlled by multiple swarms is enabled bythe innovations of the present invention, for the current discussion wewill give examples that are confined to a single swarm. This is forsimplicity of description and is not intended to limit the scope of theinnovations.

Referring again to FIG. 1, each of the computing devices 104 comprisesone or more processors capable of running the CIA routines anddisplaying a representation of the pointer 210 along with a plurality ofgraphical input choices (answer choices) 208. The computing device 104could be, for example, a personal computer running the CIA application.It could also be a mobile device such as a smart phone, tablet, headset,smart-watch, or other portable computing device running the CIA. The CIAsoftware code can be configured as a stand-alone executable or be codethat executes inside a web-browser or other shell.

While FIG. 1 shows only six computing devices 104 in communication withthe CCS 102, the system 100 is highly scalable, enabling hundreds,thousands, or even millions of users to connect simultaneously to theCCS 102, each using their own computing device 104, thereby sharing areal-time collaborative experience with the other users. In this way,large numbers of users can collaboratively control the pointer 210 togenerate a response by selecting letters, words, or numbers as a groupintelligence.

While FIG. 1 shows simple top-down architecture for direct communicationbetween the CCS 102 and each of the computing devices 104, relatedapplication Ser. No. 14/708,038 entitled MULTI-GROUP METHODS AND SYSTEMSFOR REAL-TIME MULTI-TIER COLLABORATIVE INTELLIGENCE disclosesmulti-group and tiered architectures that enable shared processing loadsamong large numbers of computing devices 104. While FIG. 1 shows adedicated CCS 102, the system 100 can be configured such that one of thecomputing devices 104 acts as the CCS 102 by running both CCS routinesand CIA routines. Such a model is generally viable only when the numberof users is low. Regardless of the architecture used, each of saidcomputing devices 104 that is engaged by a participating user includesone or more display devices for presenting a graphical user interface tothe user.

Referring next to FIG. 2, an exemplary display interface 200 is shown inaccordance with one embodiment of the present invention. Shown are aprompt bar 202, a group name 204, a target area 206, a plurality ofinput choices 208, the pointer 210, a communication menu 212, a boardselection drop-down menu 214, a physics selection drop-down menu 216, achat window 218, a chat input box 220, a current member list 222, astatistics display 224, an invite button 226, and an ask button 228.

The graphical pointer 210 is simultaneously displayed to each user bythe CIA running on his computing device 104. The pointer 210 displayedto each user appears in a substantially similar position with respect toa set of input choices 208 (as compared to the position of the pointer210 on other user's screens). The synchrony of the interfaces iscoordinated by the data 106 received by each computing device 104 sentfrom the CCS 102 over the communications link. In a current embodiment,data 106 is sent from the CCS 102 to each of the plurality of computingdevices 104 at a rate of 60 updates per second, the data 106 includingthe current position of the graphical pointer 210 (also referred to as apuck) with respect to the set of graphical input choices 208, as furthershown below.

In general, the input choices 208 are identically displayed upon all thecomputing devices 104, although some unique embodiments allow fordivergent input choices 208. For example, in some embodiments the inputchoices 208 are displayed in the native language of each user, eachinput choice 208 conveying a substantially similar verbal message, buttranslated based on a language setting of the user. This feature enablesswarms of individuals who may speak different languages and may beunable to communicate directly, to still form a swarm intelligence thatcan collaboratively answer questions or take actions. In suchembodiments, the displayed questions are also automatically translatedinto the chosen native language of the user. This is also true of adisplayed answer, and optionally the chat window 218 output.

In some embodiments, multiple graphical pointers 210 are displayed bythe computing devices 104, each of said graphical pointers 210 beingcollaboratively controlled by a different group of users. For example,500 users may be collaboratively controlling Graphical Pointer #1, whilea different group of 500 users are collaboratively controlling GraphicalPointer #2. The first group of 500 users comprises a first swarm. Thesecond group of 500 users comprises a second swarm. This unique systemand methods allow for the first swarm to compete with the second swarmin a task that is displayed simultaneously to all 1000 users on each oftheir computing devices 104.

Although multiple pointers controlled by multiple swarms is enabled bythe innovations of the present invention, for the current discussion wewill give examples that are confined to a single swarm. This is forsimplicity of description and is not intended to limit the scope of theinnovations.

As shown in FIG. 2, the CIA software running on each computing device104 is configured to display a graphical display interface 200 thatincludes at least one graphical pointer 210 and the plurality ofspatially arranged graphical input choices 208. In the example shown,the graphical pointer 210 is configured to look like a “glass puck” witha central viewing area that is transparent. In the example shown, theinput choices 208 are configured as a hexagon of six input choices 208,each input choice 208 including a graphical icon (in the embodimentshown, a dot inside a circle) and an associated word. In this case, thesix input choices 208 correspond with possible answers to questions:“Yes”, “Maybe”, “No”, “Yes”, “Bad Question”, and “No”. When the pointer210 is positioned over one of the input choices 208 such that the inputchoice 208 is substantially within a centralized viewing area of thepointer 210 for more than a threshold amount of time, that input choice208 is selected as a target. In common embodiments the threshold amountof time is 3 to 5 seconds. In the current embodiment, the centralizedviewing area appears as a graphical etching on the glass pointer 210,the etching remaining invisible until the pointer 210 approaches atarget.

As shown in the exemplary embodiment of FIG. 2, the spatially arrangedgraphical input choices 208 can comprise letters, numbers, words, and/orpunctuation marks. The input choices 208 could also comprisephotographs. In this example, if the pointer 210 is positioned over oneof the six targets for more than the threshold amount of time, thatinput choice 208 is selected as the answer to a previously askedquestion.

To ask a question, the user enters the question into the prompt bar 202.Once entered, the user clicks the ask button 228, which sends thequestion from the CIA software of that particular user (running on hiscomputing device 104) to the CCS 102. Because many users could askquestions, the CCS 102 acts as the gate keeper, deeming the firstquestion received (when no question is currently in process) as the onethat will be asked to the group. In the current embodiment, not allusers are enabled to ask questions at any given time to avoid too muchcompetition for asking. In some embodiments, credits are redeemable bythe user for the right to ask the question. In some embodiments, theuser must spend credits to ask the question, and can only ask if he hasenough credits. In some embodiments, users earn credits based on pointsawarded for participation in a session. More credits are awarded tousers who have high sync scores, less credits being awarded to userswith low sync scores.

In addition to asking questions, users can select from a plurality ofpossible target boards by using the board selection drop-down menu 214.The currently selected target board is for yes/no questions. Othertarget boards may include true/false questions, good/bad questions, andother sets of standardized answers. Also, a spelling board may beincluded where a full alphabet of input choices 208 are displayed,allowing users to spell out answers (as shown in co-pendingapplications). The spelling board may also include numbers, punctuation,backspace, blank space, and other alphanumeric characters.

As disclosed in co-pending applications, custom boards can also beentered by selecting “custom” from the board selection drop-down menu214. As will be disclosed further below, “suggestion mode” can also beselected for a given question through the board selection drop-down menu214.

As also shown in FIG. 2, users can selectively use a physics mode fromthe physics selection drop-down menu 216. As shown, a standard physicsmode has been selected, but users can choose ice mode where the pointer210 slides around on the target board as if it were frictionless ice. Agravity mode is configured to pull the pointer 210 back to a locationsubstantially near a center of the input choice set (i.e. center screen)as if by simulated gravity. In a heavy mode the pointer 210 hassubstantially higher mass than in standard mode and thus is harder forusers to collectively move. In a barrier mode, a set of physicalbarriers block a direct path to the input choices 208, forcing users tocollaboratively guide the pointer 210 around barriers to reach the inputchoices 208.

As also shown in FIG. 2, the display interface 200 includes the chatwindow 218 that allows users to exchange messages by typing in the chatinput box 220. Also included is the list of current members who are partof the group and thus enabled to ask questions and collaborativelyprovide control over the pointer 210.

Because users enter this group display interface 200 from a lobbydisplay interface where the user can choose from among a plurality ofavailable groups or swarms, the name of the current group (swarm) isalso displayed. In addition, users can invite their friends to thisgroup by clicking on the invite button 226 includes in the communicationmenu 212. In the current embodiments, these invites can leverageexisting social networks such as Facebook® friends and Twitter®followers. Also included in the interface of the current embodiment isthe statistics display 224 that gives the user of this instance of thesoftware (on this computing device 104) a listing of his personalstatistics including his score, credits, synchronicity value, the numberof rounds he has participated in, and the number of questions he hasasked the swarm.

When an exemplary question is entered by one of the users in the group,the question is sent by the CIA on that user's computing device 104 tothe CCS 102. If the CCS 102 software determines that the question isvalid, the question is then sent to all the users in the group so thatit appears substantially simultaneously on the display interface of eachof the computing devices 104. In a current embodiment, the questionappears in a large box at the top of the target board. Then a“3”-“2”-“1” countdown timer appears at the center of the target board,notifying users get ready for the collaborative answer process, orsession, to begin. The display interface (having received instructionsfrom the CCS 102) then displays a graphical “GO” and the users will thencollaboratively control the motion of the pointer 210, guiding ittowards whichever input choice 208 best satisfies the collaborative willof the group as emergent from the real-time swarm intelligence.

Each answer session is generally limited in total time by the underlyingsoftware of the present system 100, for example giving the swarm 60seconds to converge upon an answer through the collaborative motion ofthe pointer 210. This time pressure is deliberate, for it inspires usersto employ their gut instincts and intuitions rather than overthinkingthe question.

To support the use of time-pressure, the countdown clock 304 isdisplayed on a group display interface 300 of each user (as shown belowin FIG. 3), the timing of the plurality of countdown clocks 304coordinated by handshaking signals from the CCS 102. If the pointer 210does not reach the target within the allotted 60 seconds, the system 100determines that the collaboration is a failure, and sends a failureindication to the CIA of each computing device 104. In some embodiments,in response to receiving the failure indication the CIA terminating userinput and displaying the words “Brain Freeze!” on the group interface.In addition, in response to receiving the failure indication all userscould lose a number of points and/or credits for the collective failureof the group to guide the pointer 210 to a target. That said, mostswarms are successful in guiding the puck to a target. The system 100 isconfigured to deem a target achieved when the group successfullypositions the puck over a target for more than a threshold period oftime. When collaboratively controlled targeting is achieved, the answeris displayed on the CIA screens of all the users. Also displayed arestatistics for that answer—including the group sync, and that user'suser sync. Also displayed are points and/or credits awarded for thecurrent user's participation in the emergent answer.

Referring next to FIG. 3, shown is the exemplary group display interface300 of one user at a point in time during a collaboration session, i.e.after the question has been received by the computing devices 104 butbefore the collaboration session has ended. Shown are the group name204, the target area 206, the plurality of input choices 208, thepointer 210, the communication menu 212, the chat window 218, the chatinput box 220, the current member list 222, the statistics display 224,the invite button 226, a question display 302, a countdown clock 304,and a magnet icon 306.

As shown in FIG. 3, the basic layout of the display interface 300 issimilar to FIG. 2. In FIG. 3, in the target area 206 the prompt bar 202,the ask button 228, the board selection drop-down menu 214, and thephysics selection drop-down menu 216 have been replaced by the questiondisplay 302. The question display 302 appears substantiallysimultaneously upon the screens of the computers of all users in theswarm. Also displayed on the target area 206 are the set of inputchoices 208 from which the users are being asked to collaborativelyselect from. In this case the question is—“What movie should we seetonight?” and the input choices 208 include five movie names: “Jaws”,“Gremlins”, “Stand By Me”, “Indiana Jones”, and “Twister” along with asixth input choice 208, “Bad Question”. In many embodiments, the BadQuestion choice is automatically included in the input choices 208 bythe CCS 102, allowing the swarm to collectively reject the question.This allows the group not to waste time on incoherent or undesirablequestions.

After the question and input choices 208 appear on the displayinterfaces of the group members, the “3”-“2”-“1” countdown timer appears(not shown) to signal the start of the current session. When the sessionbegins, the users are now enabled to provide user input to the pointer210, guiding it towards one of the input choices 208. As the sessiontime progresses, the 60 second countdown clock 304 counts down, applyingtime pressure to the group. In FIG. 3, the countdown clock 304 is shownat 0:51, indicating that 51 seconds remain in the current session.During the current session, group members may also be inputting messagesvia text using the chat window 218, and/or may be chatting with asimultaneously enabled group voice chat. This allows interpersonalcommunication during the session.

As disclosed in the co-pending applications which have been incorporatedby reference, each user is enabled to apply forces upon the pointer 210to convey his individual intent as to how the pointer 210 should move atany moment in time. The displayed motion of the pointer 210, however, isnot a reflection of that user's individual input but a reflection of thecollectively combined group input from the entire swarm of users. Asdisclosed in co-pending applications, the collective input from theplurality of users can be such that each user's input imparts an equallyweighted contribution to the total force applied to the pointer 210. Insome embodiments, weighting factors are used to give the input forcefrom some users a higher contribution as compared to other users. Aswill be described later in this document, novel methods of adjusting theweighting factors have been developed such that computationalconfiguration of swarms can be dynamically changed over time by theunderlying software running on the CCS 102, optimizing the collaborativeperformance of a given group based on the historical performance of itsmembers.

As disclosed in the co-pending applications which have been incorporatedby reference, each user is enabled to apply forces upon the pointer 210using one of a variety of innovative methods. In one preferredembodiment, disclosed in application Ser. No. 14/738,768, each usercontrols the graphical magnet icon 306 by manipulating a mouse,touchpad, touchscreen, tilt interface, or other provided user-interfacemethod. In one such embodiment, as the user moves his mouse cursorwithin a threshold distance of the pointer 210, it turns into the magneticon 306 that grows larger in size, the closer to the pointer 210 themouse is positioned. The larger size indicates a larger force. Therelative position of the magnet icon 306, which always orients itselftowards a center of the pointer 210 under software control, indicatesthe direction of pull that user wants to impart on the pointer 210. Inthis way, a user can intuitively impart of force of a selectablemagnitude and direction upon the pointer 210.

In other embodiments, the user can tilt the portable computing device104 to convey a desired magnitude and direction. In such embodiments,the magnet icon 306 or other graphical indicator is displayed toindicate the imparted force. In some such embodiments, the user mustalso tap the screen while tilting the computing device 104, thefrequency of the taps causing a higher force to be applied. This uniqueuse of a combined tilt and tap methodology is highly effective, for itenables one handed input from users on small mobile devices. It alsoenables the ease of tilting, but avoids it feeling too passive by alsorequiring frequent tapping. In many such embodiments, the maximum forceis applied for only a short time following each tap (for example 0.5seconds) and then fades away over a subsequent period of time (forexample 3 to 5 seconds). The displayed magnet icon 306 shrinks and fadesaway along with the force magnitude. This is a highly intuitiveinterface and requires that a user repeatedly tap to maintain amaximally applied force upon the pointer 210. This is an innovativeimplementation, for it has been found that requiring frequent tappingbetter engages the user in the collaborative experience when the tiltinterface is used.

In other embodiments the user is enabled to swipe across a touchscreendisplay to indicate the magnitude and direction of the force the userdesires to apply to the pointer 210. In many such embodiments the magneticon 306 is displayed, indicative of the magnitude and directionconveyed by the swipe. In such embodiments, the swipe force is appliedfor only a short time (for example 0.5 seconds) and then fades away overa period of time (for example 3 to 5 seconds). The magnet shrinks andfades away along with the force magnitude. This is a highly intuitiveinterface and requires that the user repeatedly swipe the screen tomaintain a maximally applied force upon the pointer 210. This is aninnovative implementation, for requiring frequent and repeated swipesbetter engages the user in the collaborative experience when the swipeinterface is used.

As disclosed in the co-pending applications, the CCS 102 softwarecollects input from the plurality of users, computes a resultant motionof the pointer 210, and communicates the resultant motion of the pointer210 to each CIA of the plurality of computing devices 104. The CCS 102software also determines if the pointer 210 location is successfullytargeting one input choice 208 for more than the threshold amount oftime. If so, the CCS 102 software determines that the question isanswered and communicates the targeted input choice 208 to all membersof the group such that it is substantially simultaneously displayed uponthe display interface of each computing device 104 included in thegroup.

In this way, the system 100 of the present invention enables groups ofnetworked users to collaboratively control the graphical pointer 210 inresponse to one or more questions posed by members of group. Morespecifically, embodiments of the current system 100 enable each of theplurality of users to view on a screen of their own individual computingdevices 104, a representation of the pointer 210 and the target board,and enable each of said users to convey the user intent (also referredto as the user intent value) as to the desired direction (and optionallymagnitude) of motion that user wants the pointer 210 to move so as toselect one of the input choices displayed on the target area. The userintent is represented as a user intent vector. The user intent vectorcan be conveyed by the user, for example, by tilting his computingdevice 104 in the desired direction, swiping the screen in a desireddirection, or positioning the mouse such that the graphical magnet icon306 pulls on the pointer 210 with a desired direction.

In some embodiments, eye tracking hardware and software are included inthe computing device 104, for example the eye tracking hardware andsoftware disclosed in U.S. Pat. No. 7,429,108 to the present inventor.The CIA is configured to operate the eye tracking hardware and softwareand receive input from the eye tracking hardware are software. In thecurrent innovation, a user's gaze is tracked by the CIA and used tocompute the user intent vector that represents the user's desired motionof the pointer 210, which is communicated to the CCS 102 software. Morespecifically, the user's gaze defines a location with respect to thepointer 210. The vector between the location and the center of thepointer 210 is then used by the CIA to compute the magnitude anddirection of the user intent vector. In this way, the user can simplylook towards a direction that he desires the pointer 210 to move, andthe user intent vector is computed by the CIA and sent to the CCS 102software by the CIA. In some instances the magnet icon 306 or othergraphical element is displayed to represent the user intent vector onthe display. In this way, the user can participate in the collaborativeswarm intelligence experience using a hands-free method.

In some embodiments, a brain-computer interface (sometimes called amind-machine interface, direct neural interface, synthetic telepathyinterface, or a brain-machine interface), is employed to collect theuser input of one or more users in the swarm. In some such embodiments,the user's brain-waves are detected by the brain-computer interface ashe or she watches the pointer 210 move upon his screen. A calibrationsession is often required to correlate detected brain activity with adesired direction of motion of the pointer 210, but once thatcalibration is complete, the brain-computer interface system can be usedby the CIA to compute the user intent vector that represents that user'sdesired motion of the pointer 210 at each time-step during the session,this user intent vector being communicated to the CCS 102 software. Inthis way, the user can simply think about a direction that he desiresthe pointer 210 to move, and the user intent vector is computed and sentto the CCS 102 software by the CIA. In some instances the magnet icon306 or other graphical element is displayed to represent the user intentvector on the screen of the user's computing device 104. In this way,the user can participate in the collaborative swarm intelligence using ahands-free method.

Whatever the input method used (mouse, touchscreen, tilt, eye-tracking,or brain-tracking), the system is configured such that the user intentvector is communicated by the CIA, running on the user's computingdevice 104, to the Central Collaboration (CCS) 102. The CCS 102 collectsthe user intent vectors from the plurality of users (via their separatecomputing devices 104), and then derives a group intent vector thatrepresents the collective will of the group at that time. The groupintent vector is then used to compute an updated location of the pointer210 with respect to the target area and the input choices 208, theupdated location reflecting the collective will of the group.

In many preferred embodiments, a physical model is employed in which thepointer 210 is assigned a simulated mass and damping, each user inputrepresented as a simulated force vector. In some such embodiments, themass and damping of the pointer 210 is adjusted dynamically by thesoftware depending upon a physics mode selected by the user who askseach question by using the physics selection drop-down menu 216 shown inFIG. 2. In some such embodiments, the ice mode can be selected by theuser in which the pointer 210 glides very freely as if on ice. In somesuch embodiments, the heavy mode can be selected by the user in whichthe pointer 210 requires the collaborative pull of a large majority ofmembers of the swarm to achieve any real velocity. In some embodiments,the mass and damping are dynamically assigned by the software on the CCS102 depending upon the current size of the swarm, the larger the swarmthe higher the mass and higher the damping assigned.

Whether a physics model is used or not, the updated pointer 210 locationis then sent by the CCS 102 to each of the computing devices 104 and isused by the CIA running on each of said computing devices 104 to updatethe displayed location of the pointer 210. In this way, the plurality ofusers can watch the pointer 210 move, not based on their own individualinput, but based on the overall collective intent of the group.

As described in related U.S. patent application Ser. No. 14/668,970, thegroup intent vector can be computed from the plurality of user intentvectors as a simple average, or may be computed as a weighted average inwhich some users have more influence on the resulting collective groupintent than other users. In such embodiments, the weighting of each usercan be derived based on user scores and/or user synchronicity values(also referred to as synchrony values, sync values or performancevalues) earned during prior interactions with the system 100. In suchembodiments, each user may be assigned one or more variables thatrepresents how his or her input should be weighted with respect to otherusers in the swarm. In some embodiments the variable is called the usercontribution index and is updated regularly to reflect the skill of thatuser in providing input that helps the group reach a coherentcollaborative response. The user who demonstrates a history of“constructive input” (i.e. input that is supportive of the collectiveintent, will be assigned a higher user contribution index than the userwho has demonstrated a history of “destructive input” (i.e. input thatis substantially resistant to the collective intent). In this way, usersare incentivized push for collaborative consensus.

Those users who are supportive to the emerging consensus are determinedcomputationally by the CCS 102 by repeatedly comparing each user's userintent vector with the group intent vector. The more aligned that user'suser intent vector is with the direction of the group intent vector, themore collaborative that user is behaving. The further the user intentvector is from the direction of the group intent vector, the lesscollaborative the user is behaving. This level of collaboration isrepresented by the value defined herein and in the related applicationsas the user's synchrony (or synchronicity or sync). The synchronicityvalue may be an instant synchronicity value, i.e. one at a certaininstant in time, or may be a session synchronicity value representingthe overall user synchronicity for one or more sessions.

The synchronicity value for each individual user is determined by theCCS 102 by repeatedly comparing the user intent received from eachcomputing device 104 (representing the user input reflecting the user'sintent to move the graphical object of the pointer 210 in a givendirection) with the group intent derived from all user intents. Thesynchronicity value of the individual user is determined but computingthe difference between the user intent and the group intent. Thesynchronicity value may be an instant value, i.e., based on a singlecomparison of the user intent to the group intent at one point in time,or may be synchronicity value over a specific period of time, e.g. anaverage of the synchronicity values over that period. Thereby, the usersynchronicity value each individual user represents at least in partthat user's contribution to the collaborative control of the at leastone graphical object.

In some embodiments, each individual's synchrony value ranges between anupper bound value and a lower bound value. In one embodiment, thesynchronicity value ranges between +1 to −1, with the value +1 (theupper bound) being assigned when the user intent vector is substantiallyaligned with the group intent vector, and with the value of −1 (thelower bound) being assigned when the user intent vector is substantiallyin the opposite direction of the group intent vector, with all valuesbetween +1 and −1 being used to represent varying degrees of alignment.For example, if the user intent vector is 90 degrees out phase with thegroup intent vector, a value of 0 is assigned, for that is halfwaybetween fully convergent and fully divergent. Thus, a skilled user isone who is able to convey his individual intent as input, but do so in acooperative manner. Such a user will maintain a positive synchrony valueduring much of the session, for he or she is being supportive of thegroup intent. A user who maintains a positive value will be awarded morepoints and be assigned a higher user contribution index than a user whodoes not.

In some embodiments, the user's synchronicity values are computed as apercentage from 0% to 100%, for that is often an easier metric for usersto understand. The session synchronicity value of 100% means the userhas been perfectly in sync with the swarm. The session synchronicityvalue of 0% means the user has been entirely out of sync with the swarm.Session synchronicity values between 0% and 100% reflect relativesynchronization with the swarm, with a 50% synchronicity value meaningthe user was neutral with respect to the swarm.

Referring next to FIG. 4, shown is an exemplary display interface 400 asdisplayed on the computing device 104 being used by one user of a group,shown at a moment in time during the collaborative session after thegroup has successfully positioned the pointer 210 on one of the inputchoices 208, selecting the input choice 208 as the target, therebycollaboratively choosing the answer. Shown are the group name 204, thetarget area 206, the plurality of input choices 208, the communicationmenu 212, the chat window 218, the chat input box 220, the currentmember list 222, the statistics display 224, the invite button 226, aprefix text 402, a target text 404, a group cohesiveness scoreindication 406, a session synchronicity value score indication 408, apoints indication 410, an answer window 412, an answer options tab 414,a replay swarm icon 416, and a Tweet answer icon 418.

In this instance, the target is “Gremlins”, reflecting the swarm'scollaborative will in response to the posed question: “What movie shouldwe see tonight?” As shown in FIG. 4, the target is graphically displayedto each user on the screen of his or her computing device 104 (ascontrolled by the CIA software running on that device 104). In theembodiment shown, the graphical display includes the answer window 412including the prefix text 402 “UNUM says:” along with the chosen target:“Gremlins”.

In some embodiments, the answer is also displayed in the chat window218, as if communicated by the personified entity “UNUM” itself. Thisgives the swarm intelligence a feeling of personality and presence.

Also displayed in the answer window 412 is one or more statisticscomputed by the CCS 102 software. The statistics may reflect theperformance of the group as a whole or reflect the performance of theparticular user of that computing device 104. In this example, the groupcohesiveness score indication 406, reflecting the synchronicity of thegroup, is shown of 84%, which indicates that the group was 84% alignedin their imparted motion of the pointer 210. The group cohesivenessscore indication 406 includes the text “GROUP SYNC:” The groupcohesiveness score of 84% shows strong convergence of group members,reflecting that the swarm intelligence spoke with high “conviction” whenanswering this question. A low group cohesiveness score would reflect alow conviction for the swarm intelligence. In some embodiments the groupcohesiveness score may be repeatedly reported to and repeatedlydisplayed by each of the computing devices 104, for example during thesession.

Also displayed in the answer window 412 is the session usersynchronicity value score indication 408. The session user synchronicityvalue is a statistical indication of how well the particular user ofthis computing device 104 was aligned in his input with the swarm as awhole. The session synchronicity value score indication 408 includes thetext “YOUR SYNC:” and value of 91%. In this case, the user was veryhighly aligned, achieving a 91% synchronicity value.

Also displayed in the answer window 412 is the points indication 410,indicating the number of points earned by this user as a result of hisor her participation during the session. The user in this session hasearned 241 points, as shown in the points indication 410. The pointsindication 410 also includes the text “POINTS:”

Users earn more points (or credits) as a result of being constructivelycollaborative, helping the swarm reach a meaningful consensus. Usersearn less points (credits) as a result of being non-collaborative(obstructive), blocking the swarm from finding a meaningful consensus.In the case where the swarm cannot answer a question within the allottedtime because consensus is never reached, all users lose points(credits). This innovative scoring method encourages participants to becollaborative rather than obstructionist, thereby improving theperformance of the swarm intelligence.

Also displayed is the answer options tab 414 which gives users optionsrelated to the answer that was just reached by the swarm. The user canselectively Tweet® the answer by selecting the Tweet answer icon 418.This triggers a routine within the CIA that sends a Tweet request to theCCS 102 software, which then sends an automated Tweet to Twitter. TheTweet includes the question and the selected answer. The Tweet alsoincludes a numerical indication of the number of users who participatedin answering the given question, thus conveying the size of the swarmintelligence which produced this Tweet. The Tweet also includes ahashtag, for example “#UNUMsays”, as well as an indication of the groupcohesiveness score. In this way, the swarm intelligence system comprisedof dozens, hundreds, or even thousands of individual minds working asone can is given a unique voice as a social media entity. Enablingcollaborative groups to ask questions, answer questions, and voice theswarm's collaborative intent over Twitter as a unique entity is highlyunique and appealing to users. In some embodiments, the decision toTweet an answer is posed by the software to the swarm. A questionappears, e.g. “Should we tweet this?”, and a set of answers appear“yes”, “no”, etc. If the group picks “yes” or an equivalent, the swarmintelligence has decided to send its own Tweet. In this way, theinvention described herein enables the formation of a swarmintelligence, enables that swarm intelligence to answer questions,enables that swarm intelligence to consider the answer that emerges anddecide if that swarm intelligence wants to Tweet the answer publically.

As also included in the answer options tab 414, each individual user canselect a replay swarm icon 416. Upon selection of the replay swarm icon416, the session resulting in the current answer is replayed on thedisplay. The swarm replay is unique in that it displays an indication ofthe input of all users in the group at the same time (i.e. the swarminput), giving insight into how the swarm converged upon the collectiveanswer. The video of the swarm input is displayed in high speed(generally 2× to 5× the speed of the real session). This saves timewhile also conveying a more intuitive display of swarm activity, for thehigh speed motion of the swarm input indicates the central tendenciesmore effectively than a real-time display.

Referring next to FIG. 5, a an exemplary session replay video snapshot500 is shown. Shown are the target area 206, the plurality of inputchoices 208, the question display 302, the plurality of magnet icons306, and the replay indication 502.

As shown in FIG. 5, the session replay video includes a representationof the question asked (the question display 302), the answer choices208, and the graphical indication of the trajectory taken by the pointer210 during the answer period. As FIG. 5 is a snapshot frame of thereplay video, the pointer 210 is shown at a single location, but viewedover time the pointer 210 is shown moving across the target area 206.Also displayed is the graphical indication of the input provided by eachuser of the swarm at each time-step during the answer session. In thisinstance, the graphical magnet icon 306 is displayed for each user, thesize and orientation of each magnet icon 306 with respect to the pointer210 indicating the magnitude and direction of that user's user intentvector (magnitude and direction) upon the pointer 210 at each givenmoment in time. In this example, 8 users were participating in theswarm, collaboratively moving the pointer 210 to an answer. This methodis scalable to much larger numbers of users. The target area may alsoinclude the replay indication 502 (the text “REPLAY” as shown in FIG.5), indicating to the user that the session shown is a replay sessionand not a current session.

In some embodiments where hundreds or even thousands of users areparticipating at the same time, other innovative methods are employed tomake the replay coherent. In one such embodiment, when the number ofmagnet icons 306 exceeds a threshold, they are grouped and averaged, forexample showing one composite magnet icon 306 to represent every groupof 10 in the swarm. In this way, a swarm with 800 users can berepresented by a replay display of 80 magnet icons 306. This is highlyinformative to the user, conveying the central tendency of the swarmwithout overwhelming the display with too many magnet icons 306 (orother graphical indicators). In some embodiments, the user can selectthe replay speed.

In some embodiments, such as shown below in FIG. 6, the replay video canleave a lasting trail of plot points, allowing the user to see how thepuck moved under collaborative control. In some such embodiments, anarrow is added to the plot of points, the arrow showing the directionand optionally the magnitude of the group intent vector at each of aplurality of time-steps across the answer period.

Referring next to FIG. 6, an exemplary time history session replaydisplay 600 is shown. Shown are the target area 206, the plurality ofanswer choices 208, the question display 302, the replay indication 502,a plurality of time-step indicators 602, and a plurality of group intentvector indicators 604.

In the time history session replay display 600, a set of time-steps aredisplayed simultaneously as the plurality of time-step indicators 602,each time step indicator 602 shows the location of the pointer 210 atthat time step. The plurality of group intent vector indicators 604,shown as arrows in the exemplary display 600, are each associated withone time step and each group intent vector indicator 604 shows at leastthe direction of the group intent vector at that time step. In someembodiments, the magnitude of the group intent vector is indicated bythe length of the group intent vector indicator 604. In someembodiments, the size of the arrow of the vector indicator 604 is scaledbased on the magnitude of the group intent vector indicator 604. Thistype of innovative display allows for rapid visualization of the groupdecision process, presenting as a single image the time-history of puckmotion and group intent. In the example display 600 above, thetime-history reveals that the group initially considered Marshawn Lynchas the non-quarterback NFL player who would earn the most touchdowns,but the swarm could not reach consensus during the real-time closed-loopcollaborative session and instead transitioned the collective willtowards Gronkowski. Thus, rather than just seeing the answer thatemerged from the group, this type of plot allows a viewer to understandthe deliberation process within the “group mind” that emerged.

In some embodiments, the CIA software displays a answer options tab orbutton, in some embodiments similar to the answer options tab 414 shownin FIG. 4, allowing users to post the replay display image toInstagram®, Twitter, Facebook®, or other social media site. The sharingof such a plot is highly valuable because the single image reveals notjust the decision of the collaborative swarm but the deliberationprocess that produced the decision.

Referring next to FIG. 7, a snapshot of an exemplary pointer motionrepresentation session replay video 700 is shown. Shown are the targetarea 206, the plurality of input choices 208, the question display 302,the replay indication 502, a pointer representation 702, and a pointermotion tail 704.

As in FIG. 5, the snapshot 700 represents a point in time during thesession replay video. In the embodiment of FIG. 7, the location of thepointer 210 at the current timestep is represented by the pointerrepresentation 702. In this embodiment the pointer representation 702 isa multipointed star shape, although it will be understood that thepointer representation 702 and the pointer motion tail 704 may be anyshapes, sizes and/or colors suitable for conveying the pointer motioninformation. Motion of the pointer 210 over a defined range of previoustimesteps is represented by the pointer motion tail 704. The pointermotion tail 704 indicates the motion of the pointer 210 immediatelyprior to the current pointer location. In the embodiment shown, eachtimestep is represented by the same multipointed star shape as for thepointer representation 702, with each newer timestep overlapping theprevious timestep, and the pointer representation 702 in front. Thegraphical effect is similar to a “worm”, as the pointer motion tail 704follows the pointer representation 702 as older timesteps disappear andnewer timesteps are added. The pointer motion during the replay is inthe form of an animated moving shape with the pointer motion tail 704representing the past motion of the pointer 210 at the given moment intime. The pointer motion tail 704 is animated to stretch longer when themotion is faster and compress shorter when the motion is slower. Themotion during the replay video makes the pointer motion representationresemble a worm or a snake or a slime mold.

There is a significant innovative use for the pointer motionrepresentation session replay video, for the motion of the simulatedorganism across the screen tends to anthropomorphize the data, helpingthe users see the collective will of the group as a living thinkingcreature as they watch the replay in this form. In the exemplary pointermotion representation session replay 700 of FIG. 7, video the animatedpointer motion representation “slithers about” the target area 206 basedon the previously stored pointer position data and stored group intentvector data of the session, resulting in the the pointer motionrepresentation appearing like a creature that is considering the optionsand making a decision.

In some embodiments, the software includes the share button (in someembodiments similar to the an answer options tab 414), allowing users topost this animated slime mold replay as a video to Vine®, Instagram®,Twitter®, Facebook®, or other social media sites.

The sharing of such a video is highly valuable because the animationreveals not just the decision of the collaborative swarm but allowsusers to view it as the Emergent Intelligence of a new entity.

Referring next to FIG. 8, an exemplary discrete image target area 800 isshown. Shown are the target area 206, the question display 302, thecountdown clock 304, the pointer 210, the magnet icon 306, and theplurality of answer choices 208.

In the discrete image target area 800 of FIG. 8 the answer choices 208are represented as images. For example, one answer choice 208 comprisesan image of a strawberry, representing the answer choice of“strawberry”. In some embodiments the answer choices 208 comprise video.As with previous target areas, each answer choice 208 is associated witha graphical target, whereby selecting of the graphical target using thecollaborative process selects the associated answer choice 208.

The group collaboratively selects among the discrete image answerchoices 208 by working together in real-time to move the pointer (puck)210 to the graphical target associated with one of the images. Thissystem is functional, but does present some new issues to be resolved.Most significantly, the system requires the storage and message passingof images or videos, which slows communication time. For the embodimentwhere each answer choice 208 image is passed separately, even if theimages are referenced as URLs, this requires a plurality of URLs to bepassed to each computing device 104 for each question that involves theplurality of answer choice images (or videos). This also requires theasker of the question to identify the plurality of answer choice 208images and specify their location around the target area 206 when askinga question. For users, this can involve identifying a plurality of imageURLs which can be time consuming. And finally, it means that data mustbe stored for all images in a database if we want to allow users toreplay questions at a later date.

To provide a more efficient and more flexible model for image selection,an alternate paradigm has been developed wherein a single target areaimage 1102 is presented to the users, referenced by a single URL, saidimage including a plurality of possible answer choice 208 images withinit, as shown below in FIG. 11. Selection of the answer choice imageitself constitutes selection of the target. In some embodiments, thatsingle target area image 1102 may include a number of discrete answerchoice 208 images, (such as food images, logos, product images, etc.),wherein the group selects one of the answer choice 208 images includedin the image 1102. The answer choice 208 images are separated spatiallysuch that they can be individually targeted by the collaborativelycontrolled pointer 210. In other embodiments, the single target areaimage 1102 may include a calendar with a set of discrete boxes for days,which can be each be individually targeted by the collaborativelycontrolled pointer 210. In other embodiments, the single target areaimage 1102 may include an analog range of possible target areaselections—for example, a graphical map wherein any pixel is a possiblytargeted location on the map. Thus, by presenting a single image withflexible target locations, it becomes much easier to ask the question.

To select one answer choice 208 within the target area image 1102, thecollaborative swarm may be enabled to place a graphical pin 1402 orother graphical location indicator on a chosen location upon the singletarget area image 1102. The location of this graphical pin 1402represents the chosen target location decided upon by the collaborativeswarm through the real-time synchronous control of the pointer 210 (i.e.puck). The pin 1402 is then displayed upon the target area image 1102 asa means of presenting the target answer to the question.

Referring next to FIG. 9, a flowchart of a method for location selectionby motion threshold is shown. Shown are a motion threshold step 900, atime threshold decision point step 902, a select position step 904, anda position not selected step 906.

The method for location selection by motion threshold occurs during thecollaborative session while the users are actively directing the pointer210, with the group intent vector represented on the target area 206 bythe motion of the pointer 210 across the target area 206 as described inthe related applications. Two values are previously defined and storedby the CCS 102: a motion threshold value and a time threshold value.

In some embodiments, the motion threshold is a speed, i.e. a distanceper time. If the speed of the pointer during the collaborative sessionis less than the motion threshold value, it is determined to have fallenbelow the motion threshold. In one embodiment the distance is defined asa number of pixels. For example the motion threshold value could be 5pixels per second. Thus if the pointer's motion falls below the rate ofchange of 5 pixels per second (i.e. speed), the pointer motion isdetermined to have fallen below the motion threshold.

The time threshold value is a period of time for which the pointer speedmust be below the motion threshold value. The time threshold value couldbe, for example, 4 seconds. If the pointer speed drops below the motionthreshold value for more than the time threshold value amount of time,the CCS 102 determines that the collaborative group selects the currentpointer location as the collaborative decision.

In the first motion threshold step 900, the CCS 102 is running analgorithmic process to determine if the pointer speed has fallen belowthe motion threshold value, and has determined that the pointer speedhas fallen below the motion threshold value. The method then proceeds tothe next time threshold decision point step 902.

In the time threshold decision point step 902, the CCS 102 determineswhether the pointer speed has fallen below the motion threshold valuefor a period of time more than the time threshold value. If the pointerspeed falls below the motion threshold value for more than the timethreshold value, the method proceeds to the select position step 904 andthe current pointer position is selected as the target.

If the pointer speed has not fallen below the motion threshold for theperiod of time, the method proceeds to the position not selected step906, and the pointer position is not selected, and the collaborativesession continues.

Referring again to FIG. 9, two threshold values are defined for use inthe location selection by motion threshold process, one being the motionthreshold value and one being the time threshold value. The algorithmicprocess is then run by the CCS 102 software to determine if thepointer's collaboratively controlled speed falls below the motionthreshold value for more than the time threshold value amount of time.

Of course, falling below the motion threshold is not enough to indicatethat the group desires to select a particular location. That's becausethe speed of the pointer could simply be changing direction and thuspass through a rate-of-change that falls below the motion thresholdvalue as the puck slows down, stops, then proceeds in a differentdirection. This is why we also employ the time threshold value

The time threshold value is the period of time for which the motionthreshold must be met. The time threshold value could be, for example, 4seconds. Thus, if the pointer speed drops below the motion thresholdvalue for more than the time threshold value amount of time, the CCS 102determines that the collaborative group desires that the pointerlocation is selected as the collaborative decision. This could be, forexample, the set of values such that the puck motion needs to fall below5 pixels per second for more than 4 seconds.

Referring next to FIG. 10, a flowchart of a method for locationselection by force threshold is shown. Shown are a force threshold step1000, the time threshold decision point step 1002, the select positionstep 904, and the position not selected step 906.

In another embodiment, the CCS 102 assesses a group intent vector,wherein the group intent vector is a magnitude and direction derivedfrom all the user inputs indicating desired movement of the pointer, aspreviously described in the related applications. If a magnitude of thegroup intent vector falls below a force threshold value for more thanthe time threshold value amount of time, it is determined that thecollaborative swarm of users desire to target the current pointerlocation. In some embodiments, the force threshold value is defined as apercentage of the total possible force magnitude that can be applied bythe collaborative swarm if all users were pulling in the same directionat the same time. The force threshold value could be, for example 5% ofthe maximum possible force magnitude. The time threshold could be, forexample, 4 seconds.

In the first force threshold step 1000, the CCS 102 is running analgorithmic process to determine if the group intent vector magnitudehas fallen below the force threshold value, and has determined that thegroup intent vector has fallen below the force threshold value. Themethod then proceeds to the next time threshold decision point step 902.

In the time threshold decision point step 1002, the CCS 102 determineswhether the group intent vector magnitude has fallen below the forcethreshold value for a period of time more than the time threshold value.If the group intent vector magnitude falls below the force thresholdvalue for more than the time threshold value, the method proceeds to theselect position step 904 and the current pointer position is selected asthe target.

If the group intent vector magnitude has not fallen below the forcethreshold for the period of time, the method proceeds to the positionnot selected step 906, the pointer position is not selected, and thecollaborative session continues.

Referring again to FIG. 10, in one example the force threshold value is5% of the maximum possible force, and the time threshold value is 4seconds. The CCS 102 determines that the group of collaborativelycontrolling users wishes to target a location with the graphical pointerif the group intent vector applied to the pointer falls below 5% of themaximum possible force for more than 4 seconds. This method is highlyeffective because it accounts both for users who choose to stop pullingas well as users whose pulls cancel out with other users. This getsdirectly at the collaborative will of the swarm as its own unique anddistinct intelligent entity.

In some embodiments, the threshold values used in the location selectionby motion threshold method and the location selection by force thresholdmethod above are not fixed, but change during the time period of aresponse. More specifically, the threshold values used in the locationselection by motion threshold method and the location selection by forcethreshold method above can be inventively configured to be increasedover time, during the period of a response. This is highly effective atensuring that a group comes to a decision, for as the period ofindecision lingers, the thresholds for determining selection areincreased.

In some embodiment, the motion threshold value increases, with theincrease starting at the beginning of the collaborative session or theincrease starting when a time period has elapsed during the currentsession. For example, in one inventive embodiment, the motion thresholdused by the CCS 102 software to determine if the group has selected alocation on the single image 1102 starts at a minimum value and isincreased over time during the selection period until reaching a maximumvalue. In some such embodiments, the increase happens linearly overtime. In other embodiments, the increase happens at discrete timeintervals. For example the motion threshold could be configured with aninitial maximum value of 5 pixels per second. The software can then beconfigured to start increasing that threshold after the time period ofthe collaborative decision process has exceeded an elapsed time of 30seconds. Then, during the next 30 seconds, the motion threshold can beconfigured to be increased by 1 pixel per second every 5 seconds.

This method allows the group to deliberate collaboratively, requiringthe pointer 210 in the initial stage of the collaborative session toreduce the pointer speed to less than 5 pixels per second in order tocollaboratively make a decision. But, if the group is having a hard timemaking a decision, as indicated by the elapsed time of the responseexceeding a certain period (e.g. 30 seconds), the motion threshold fordetermining a collaborative decision is eased. This easing of the motionthreshold will allow the group a better chance of reaching an identifiedcollaborative decision. The same process can be inventively employed inlocation selection by force threshold method increasing the forcethreshold value. The same process can be inventively employed on thethreshold time value, but instead of increasing the time thresholdvalue, the CCS software is configured to decrease the time thresholdvalue across the elapsed time of the collaborative decision process. Insome embodiments, the CCS 102 software is configured, upon determiningthat the collaborative decision process has lingered beyond a timeinterval, such as 30 seconds, to increase the simulated damping in thephysics model for puck motion. This assists the collaborative group inconverging upon a decision, for the increased damping resists the straymotion of the puck 210, allowing the puck 210 to settle on an answereasier.

Referring next to FIG. 11, an exemplary image target area 1100 is shownprior to the start of the collaborative session. Shown are the targetarea 206, the question display 302, the pointer 210, the target areaimage 1102, and the plurality of target images 1104.

As described previously is FIG. 8, the target area image 1102 may beused to provide greater target selection flexibility, instead ofselection being limited to the answer choices arrayed on the target area206, for example in the hexagon configuration previously described.

As shown in the exemplary image target area 1100 of FIG. 11, the singleimage 1102 is displayed in the target area 206 that allows for thepresentation of the number of discrete target images 1104 within theimage 1102. The image 1102 is shown bounded by a rectangular box forillustration purposes, but typically an extent of the image 1102 withinthe target area 206 will not be graphically displayed on the target area206.

In the exemplary image target area 1100, the target images 1104 includedare illustrations of fruit, similar to the answer choices 208 shown inFIG. 8. But here, instead of needing multiple image files and multipledesignated answer choices 208, the image 1102 can be presented by theuser without any representations being conveyed as to the content of theimage or the location of the answer choices 208. The CCS 102 software,in fact, needs not to convey anything about locations to the computingdevices 104—the CCS 102 software just needs to send the image 1102 tothe plurality of computing devices 104 for display, send therepresentation of the question to the group, and moderate thecollaborative session, whereby the group places the pointer 210 on alocation on image 1102, which becomes the target. The meaning of thetarget is inherent in the understanding of the users participating.

As shown in the FIG. 11, the target area image 1102 is presented tousers on their portable computing devices 104. This is generallyachieved by a single URL being included with the question, the URLproviding a link to the single target area image 1102. The image 1102 isdisplayed on the screens of the users, along with the collaborativepointer 210, and along with the question display 302, “What fruit shouldwe serve for dessert?” The image 1102 includes the plurality of targetimages 1104, distinct graphical objects that are known to the users asunique objects that can be selected among. This said, the software neednot know the location of the target images 1104 within the single image1102. This makes the method highly efficient from a processing andcommunication perspective. The pointer 210 is generally displayedinitially at the center of the image 1102 (as shown in FIG. 11), in anopen area of the target area image 1102. Thus it is up to the user tochoose or craft the target area image that has an empty area in thecenter where the puck 210 would start (or other suitable area within theimage 1102).

In general, a “3, 2, 1” countdown is presented to all users prior to thestart of the collaborative session, as disclosed in the priorapplications. Then the users collaboratively control the pointer 210,for example using the graphical magnet icon 306 as previously disclosed.In many preferred embodiments, each user only sees his or her own magneticon 306, while watching the collectively influenced motion of the puck210. In other embodiments, each user can see their own magnet icon 306,as well as ghosted images of other user's magnet icons 306. Thisgenerally appears as a real-time swarm of magnets, the motion and flowof the swarm being highly informative.

Referring next to FIG. 12, an exemplary image target area 1200 is shownduring a collaborative session. Shown are the target area 206, thequestion display 302, the pointer 210, the target area image 1102, theplurality of target images 1104, the magnet icon 306, and the ghostedmagnet icons 1202.

During the collaborative session, the pointer 210 is in motion undercollaborative control of the real-time group of users participating inthe closed-loop dynamic system. In the embodiment shown in FIG. 12, themagnet icon 306 indicates the intent of the user, while the ghostedmagnet icons 1202 indicate the intents of the other users of the group.

As shown in FIG. 12, the pointer 210 is in motion over the single targetarea image 1102 which includes the number of discrete target images 1104or locations that are knowable to the users (in this case images offruits). The users are pulling the pointer 210 collaboratively throughreal-time closed-loop control, each user applying a force upon thepointer 210 through their own control of the graphical magnet icon 306as shown in FIG. 12.

It should be noted that FIG. 12 represents a single time-step across ananswer period that might take 10 to 60 seconds of pointer motion. TheCCS 102 software is monitoring pointer motion during this time,determining if the users desire the pointer 210 to target a particularlocation within the target area image 1102. This is determined, forexample, using either the selection method of FIG. 9, the selectionmethod of FIG. 10 or a combination thereof.

At each particular time-step, the CCS 102 software determines whetherthe target selection criteria has been met either the selection methodof FIG. 9 of the selection method of FIG. 10 (or both, if used incombination). This happens when the pointer 210 is in a position, forexample, as shown in FIG. 13 below.

Referring next to FIG. 13, an exemplary image target area 1300 is shownafter a target image has been selected by the collaborative session.Shown are the target area 206, the pointer 210, the question display302, the target area image 1102, the plurality of target images 1104, atarget point 1302, and a coordinate origin 1304.

As shown in FIG. 13, the group has selected the target point 1302 of thetarget area image 1102. The selected target point 1302 is such that aparticular target image 1104, in this case a strawberry image, islocated within the lens of the pointer 210. But because of the highlyefficient nature of this method, the CCS 102 software need not have anyknowledge (i.e. data) indicating that there is a particular objectdisplayed at the target point 1302. This is evident to the users, whouse their visual senses to all come to that same conclusion, but thesoftware need not concern itself with that fact. Instead, the softwaresimply knows that the users have targeted the target point 1302, definedby an X and Y coordinate, within the image 1102. This is all that thesoftware needs to record and convey to the plurality of computingdevices 104.

Thus, the CCS 102 software determines that the pointer 210 has targetedthe target point 1302 with coordinates (TargetX, TargetY) within thetarget area image 1102. The target point 1302 coordinates can bereferenced with respect to the coordinate origin 1304 of the image 1102,for example an upper left corner of the image 1102, referenced as 0,0.

The CCS 102 software then communicates to the computing devices 104,that the target point 1302 has been selected by the group and that thetarget point 1302 is at the X-Y coordinate location (TargetX, TargetY).

The CIA software running on the computing devices 104 of each user thendisplays the graphical pin 1402 or other graphical indicator on thedevice 104, showing the location of the target point 1302 on the image1102, as shown in FIG. 14 below. Because the users make their own visualinterpretation of the image, they know it corresponds to a strawberrybut the software need not have any data to that effect. This makes theprocess highly efficient.

Referring next to FIG. 14, an exemplary image target area 1400 with thegraphical pin 1402 is shown after the target point 1302 has beenselected by the collaborative session. Shown are the target area 206,the pointer 210, the question display 302, the target area image 1102,the plurality of target images 1104, and the graphical pin 1402.

The graphical pin 1402 is displayed upon image 1102 at the target point1302 that was collaboratively selected by the real-time synchronousswarm of users. This indicates to all users that a strawberry wasselected as the answer to the question, although the software need notknow what was displayed at that portion of the single image 1102. Thismakes it highly efficient.

The CCS 102 software then stores information about the session,including the question asked, the image 1102, and the target point 1302X-Y coordinates (TargetX, TargetY) in its database, so that future userscan access the question and see the answer. The answer is displayed tofuture users as the image 1102 plus the graphical pin 1402 located at(TargetX, TargetY) coordinates upon the image 1102, referencing theknown coordinate origin 1304. This allows for efficient data storage.

The method above is not just efficient, it's highly flexible. Inaddition to allowing one single image to include collections of discretegraphical objects like the fruits above, the method can be used toidentify a location on one single image that is not comprised of uniqueand discrete objects but instead a continuum of selectable positions.For example, a group of engineers working on the design of a mechanicalpart could collaboratively work as swarm intelligence to identify alocation of the weakest point on a graphical drawing of that mechanicalpart using methods and systems disclosed herein.

Other inventive configurations include the display of graphical maps orcalendars such that a group of users can function as a swarmintelligence to collaboratively select a location on a map or a date ona calendar in response to a question posed to the group.

Referring next to FIG. 15, an exemplary map image target area 1500 isshown prior to the start of a collaborative session. Shown are thetarget area 206, the question display 302, the pointer 210, and thetarget area image 1102.

As shown in the exemplary map image target area 1500 of FIG. 15, thesystem 100 enables one user to ask a question to the group ofcollaborative participants in the real-time synchronous swarm, saidquestion including a textual prompt and the selection by the user of onemap image. In preferred embodiments, the map image is selected by theuser from a plurality of available map images. For example, the userchooses a USA map from among a plurality of other locative maps such asworld maps and other country maps. In this example, the user asks “Whereshould we focus our Marketing Campaign?” (as shown in the questiondisplay 302) to a group of users who comprise a marketing team and whowill tap their collaborative intelligence to provide an answer. Duringthe subsequent collaborative session the group will then collaborativelycontrol the pointer 210, collaboratively moving it over the image 1102.Using the methods described in FIGS. 9 and 10 above, the CCS 102software determines when group has selected the target point 1302 on themap target area image 1102. Once collaboratively selected by the swarm,the graphical pin 1402 or other identifier is displayed on the image1102 indicating the collaboratively selected location.

Referring next to FIG. 16, an exemplary map image target area 1600 isshown after completion of the collaborative session of FIG. 15. Shownare the target area 206, the question display 302, the target area image1102, and the graphical pin 1402.

Once the collaborative decision process is complete, the CCS 102 sendsthe coordinate location of the target point 1302 to the computingdevices 104, and the graphical pin 1402 is displayed on the image 1102,indicating the target point location. The image 1102, the target point1302 coordinate location and the question are stored in a database forlater review. This data allows for a complete display of both questionand response by simply positioning the graphical pin 1402 at the targetpoint location on the image 1102.

Again, this is highly efficient because the CCS 102 software need notstore information related to the relative location of particular placeson the map, for the users can interpret that themselves based on thelocation of the graphical pin 1402 on the image 1102. This allows forefficient data passing and efficient data storage, as well as a highlyversatile means for users to post pictures of all kinds as parts ofquestions without needing to characterize the pictures spatially. Theusers can use their shared interpretation of the picture content to dothat.

Referring next to FIG. 17, an exemplary calendar image target area 1700is shown during a collaborative session. Shown are the target area 206,the question display 302, the pointer 210, the magnet icon 306, thetarget area image 1102, a forward arrow 1702 and a backward arrow 1704.

In the exemplary calendar target area 1700 of FIG. 17, the systemenables one user to ask a question to the group of collaborativeparticipants in a real-time synchronous swarm, said question includingthe textual prompt (as shown in the question display 302) and theselection of one calendar image 1102. In this example, the user asks“When should we plan our next practice?” to a group of users whocomprise a softball team and who will tap their collaborativeintelligence to provide the answer. The group collaboratively controlsthe pointer 210, positioning it over a target location on the calendartarget area image 1102 selected by the question asker. Using theselection method of FIG. 9 and/or the selection method of FIG. 10 asdescribed above, the CCS 102 software determines when the group hasselected one target point 1302 on the calendar image 1102. Oncecollaboratively selected by the swarm, the graphical pin 1402 or otheridentifier is displayed indicating the collaboratively selectedlocation. The users know this location corresponds with a particulardate.

In some embodiments the users can also collaboratively control the puck210 to land on the forward arrow 1702 or the backward arrow 1704, bothincluded in the target area 206. The forward arrow 1702 and the backwardarrow 1704, when selected, change the month of the calendar to a nextmonth or previous month respectively. In the exemplary calendar targetarea 1700 of FIG. 17, the forward arrow 1702 and the backward arrow 1704are positioned below the calendar target area image 1102.

As with the prior disclosed methods, the single image 1102 (in this casea calendar) are saved along with the selected target point 1302 and theposed question, providing a complete record of the question and answerthat can be reviewed by users. Again, this is highly efficient becausethe software need not store information related to the relative locationof particular days on the calendar, for the users can interpret thatthemselves based on the location of the graphical pin 1402 on the singleimage 1102.

Referring next to FIG. 18, a flowchart for a method for collaborativezooming is shown. Shown are a receive zoom indications step 1800, a zoomchange step 1802, and an apply zoom step 1804.

In some embodiments, the system allows a real-time swarm ofcollaborative users to not only identify the target point 1302 on theimage 1102 but selectively zoom the image 1102 under synchronous groupcontrol. To support this, a novel yet intuitive collaborative userinterface method had to be developed.

The collaborative zoom method of FIG. 18 involves defining a zoom-inregion and a zoom-out region upon, affixed, or otherwise associated withand moving with the graphical pointer 210 that is responsive tocollaborative pull from the plurality of users. By having the zoom-inregion and the zoom-out region move with the graphical pointer 210, wemaintain user focus and interaction with the pointer 210 (puck). This isessential because there is the plurality of users interacting with thepointer 210 at any one time, so if users disengage the puck 210 toengage a separate zoom feature, other users who have not disengaged willkeep moving the puck 210, thus disrupting the true collaborative natureof control. This is the unique challenge for collaborative interfaces:to maintain all users engaged with the puck 210 as they transition fromexpressing the locative pull to expressing image zoom.

In one such embodiment, the zoom-in and zoom-out regions are onlydisplayed and active when the motion of the puck 210 falls below thethreshold speed and/or the group intent vector falls below the thresholdforce level. This ensures that zoom-in or zoom-out only happens when thepuck 210 is approximately still, allowing the users to focus on theimage zoom rather when image location is not changing significantly.

As for the display of the zoom-in and zoom-out regions, in preferredembodiments the regions move with the puck 210, always staying in closeproximity to the puck 210, either close outside an outer boundary of thepuck 210 or actually within the outer boundary of the puck 210. In someembodiments, the placement with respect to a center of the puck 210 isdependent upon the user's magnet icon 306 location with respect to thepuck 210. For example, if the user's magnet icon 306 is above the puck210, pulling upward, the zoom-in and zoom-out regions can be displayedabove the puck 210 too, not far from the user's magnet icon 306.Similarly, if the user's magnet icon 306 is to the left of the puck 210,pulling leftwards, the zoom-in and zoom-out regions can be displayed tothe left of the puck 210 as well, not far from the user's magnet icon306. This allows the user to maintain his or her desired pull, and withvery little change in motion, target either the zoom-in or zoom-outregions. In many embodiments, these regions are annotated with astandard convention such as “+” for zoom-in and “−” for zoom-out, whichis generally well known in the art.

In the first receive zoom indications step 1800, the CIA softwarerunning on each computing device 104 determines if the user's graphicalmagnet icon 306 (or other user intent vector indicator) is within or onone of the zoom-in or zoom-out regions. If so, the CIA software runningon the user's computing device 104 sends a signal to the CCS 102software, representing that said user is making a zoom indication duringthe present time-step and indicating whether that zoom indication is azoom-in or a zoom-out indication.

In one such embodiment, the CIA software sends a zoom indicationvalue=+1 if the user is indicating zoom-in, a zoom indication value=−1if the user is indicating zoom out, and sends a zoom indication value=0if the user is not indicating any zoom. At each of a plurality oftime-steps of the session, the CCS 102 software receives the zoomindication values from the plurality of users in the collaborative swarmvia each of their computing devices 104 over said communication network.

The method then proceeds to the zoom change step 1802. During the zoomchange step 1802, the CCS 102 software is operative to compute asummation, average, and/or median value of the plurality of receivedzoom indication values from the users.

Based upon the summation, average, and/or median value of the pluralityof zoom indication values received from the plurality of synchronoususers in the swarm, the CCS 102 software determines if the single image1102 should be zoomed in, zoomed out, or unchanged in zoom for this timestep. In response to this determination, the CCS 102 software sends azoom signal to each of the plurality of computing devices 104,indicating if the single image 1102 should be zoomed in, zoomed out, orunchanged in zoom for the current time-step. Because these values aretime-step related, they are generally conveyed as a delta zoom (i.e. anincremental change in zoom). In some such embodiments, the delta zoom iseither a +1%, −1%, or 0% change in zoom for the current time step.

In response to the delta zoom value received from the CIA softwarerunning on each computing device 104, the CIA software on each device isoperative to change the currently displayed zoom level of the singleimage 1102 by the defined delta zoom, in the final apply zoom step 1804.Thus if the delta zoom received from the CCS 102 software by each of theCIA clients running on the computing devices 104 was +1%, the CIAsoftware would zoom-in on the single image 1102 by +1% during thistime-step. If another +1% was received by the CIA software in the nexttime-step, the CIA software would zoom the single image 1102 by another+1% in the next time step.

It is important to note that substantially the same delta zoom value issent to all computing devices 104 in a substantially simultaneous mannerbecause this is a synchronous real-time closed loop system where allusers are responding in real-time to the same change in zoom, andthereby changing their input for future time-steps. In this way, theusers will be viewing the zoom synchronously, all of them adjustingwhether they are targeting the zoom-in, zoom-out, or no zoom indicatorsbased on how the image 1102 is zooming. In this way, theycollaboratively form a zoom-controlling swarm that can converge inreal-time upon a collectively desired zoom level. The process is fastand seamless, everyone feeling consequential. Thus, the users'perceptual system is closing the loop on one end, while the CSS 102 isclosing the loop on the other end, allowing for all users to work as oneas a real-time swarm.

The CCS 102 software also monitors maximum and minimum zoom limits suchthat the CCS 102 does not allow for delta zooms to be sent to computingdevices 104 such that the delta zoom would drive the total current zoomlevel of the single image 1102 above the maximum zoom limit or below themin zoom limit.

In some embodiments the CCS 102 maintains a running average of zoomindication values received from the plurality of computing devices 104across a series of time-steps. For example, if the server 102 isreceiving data from the portable devices 104 at a rate of 60 reports persecond, the running average could be set to compute the average zoomindication tally received across the last 6 time steps. This would giveone average tally every tenth of a second. This helps smooth the zoomacross user input, but is still fast enough on a human-time scales thatit functions as a real-time closed loop system.

In alternate embodiments, collaborative zoom is accomplished by theplurality of users in a very different but also effective way. In suchalternate embodiments, a zoom-in and zoom-out region is placed upon thetarget area 206 and is not engaged by the magnet icon 306 (as in theembodiments above) but instead is engaged by the pointer 210 (puck)itself. Thus, the users must collaboratively control the puck 210 toposition it upon or within the designated zoom-in or zoom-out region onthe target area 206 so as to cause the displayed image 1102 to zoom inor zoom out accordingly. Thus, if the collaborative swarm makes thecollective decision to zoom in on an image 1102 (such as a map), they doso by moving the puck 210 onto or within the designated zoom-in area onthe screen through the same real-time collaborative control as theywould move the puck 210 onto other targets. This allows for simpleimplementation but does have a drawback compared to the priorembodiments, for it causes the puck 210 to leave the other activity itmay have been engaged in, for example targeting locations on an image,to instead target the zoom-in or zoom-out region. This makes for aslower interface than the magnet controlled embodiments.

Referring next to FIG. 19, a panning-enabled map target area sequence isshown. Shown are the question display 302, the target area 206, thepointer 210, the target area image 1102, a first map target area 1900, apanning boundary 1902, a group intent direction arrow 1904, and a secondmap target area 1906.

In some embodiments, the system 100 allows the real-time swarm ofcollaborative users to selectively pan the image 1102 or set of imagesthat are larger than the target area 206. For example, a large map inwhich areas of the map are beyond the limits of the target area 206requires panning to view the hidden locations. To enable thecollaborative swarm of users to pan the image 1102, a novel method hasbeen devised in which the graphical puck 210, when hitting up againstthe panning boundary 1902, is stopped from moving. In the embodimentshown in FIG. 19, the panning boundary 1902 coincides with a displayextent of a portion of the image 1102, but the panning boundary 1902 mayhave a different shape and/or location. If the users collaborativelycontinue to pull against the panning boundary 1902 that is stoppingthem, the image 1102 is panned in the opposite direction to thecollaborative pull. In preferred embodiments, a speed of the panning ofthe image 1102 in said opposite direction is based at least in part uponthe group intent vector imparted by the swarm. Thus the magnitude of thenet pull on the puck 210 by the users is used to control the speed ofpanning such that the larger the net force pulling against the panningboundary 1902, the faster the panning in the inverse direction. Thisgives the collaborative group an intuitive proportional control ofpanning speed. And because the puck 210 can be pulled up against allfour of the panning boundaries 1902, it also gives control overdirection. In the corners, the puck 210 can be used to pull against twopanning boundary 1902 edges at once, each proportional to the pullagainst the panning boundary 1902, thus allowing for diagonal panning.As shown in FIG. 19, the first map target area 1900 includes a portionof the larger image. In the example for FIG. 19, the larger image is amap of the U.S. and the image 1102 is displaying a portion of the largermap image, with the rest of the map outside the limits of the displayedtarget area image 1102. The first map target area 1900 includes thepointer 210, which has been collaboratively pulled against the panningboundary 1902, in this example the right-hand edge of the rectangularpanning boundary 1902. The group intent vector is indicated by the groupintent direction arrow 1904, and is oriented horizontally and pointed tothe right, indicating a right-hand directional intent. It will beunderstood that the group intent directional arrow is for understandingof the group intent vector only, and would not normally be displayed onthe computing devices 104. The second map target area 1906 shows thepanned target area 206 as a result of the pointer 210 location andconcurrent group intent vector. The image 1102 has panned across thelarger map image in an opposite direction to the direction of the groupintent direction arrow 1904, i.e. to the left.

To support collaboratively controlled panning among the plurality ofusers engaged in real-time synchronous control, the method has beendevised in which the collaboratively controlled puck 210 is preventedfrom exiting the panning boundaries 1902, instead butting up against theedges. As shown in the first map target area 1900, the puck 210 has beenpulled under collaborative control from a center of the displayed image1102 to the rightmost edge, butting up against a limit visually depictedby a right edge of the panning boundary 1902. If the users continue toapply collaborative force to the puck 210, instead of the puck 210moving past the panning boundary 1902, the CCS 102 and CIA software areconfigured to instead pan the image 1102 in the opposite direction tothe collaborative pull of the puck 210. Thus if the users arecollaboratively pulling the puck 210 rightward with a group intentvector that opposes the panning boundary 1902, the CIA and CSS 102software work in combination to pan the image 1102 on all user screensin the opposite (leftward) direction. Further, the speed of panning inthe leftward direction is implemented such that a panning magnitude isproportional to the magnitude of the group intent vector in therightward direction. If the users are pulling on the puck 210 at anangle that's not directly orthogonal to the panning boundary 1902, onlythat component of force of the group intent vector is used to controlthe leftward panning.

In this way, users can move the puck 210 under collaborative controlacross the image 1102 and when the puck 210 reaches a limit of travel(for example the panning boundary 1902), the puck 210 is prevented frommoving further in that direction of travel, the CCS 102 software beingconfigured instead to pan the image 1102 in the opposite direction, witha speed proportional to the magnitude of the collaborative group intentvector as it pulls against the barrier to motion. And because the puck210 can be pulled up against all four of the panning boundary 1902 edgesof the image, it also gives the collaborative swarm real-time controlover the direction of panning. To pan the image 1102 rightward, thecollaborative group is enabled to pull the puck 210 against theleft-side edge of the panning boundary 1902, the stronger thecollaborative pull against the left-side edge the faster the rightwardpanning. To pan the image 1102 upward, the collaborative group must pullthe puck 210 against the bottom edge of the panning boundary 1902, thestronger the collaborative pull against the bottom edge the faster theupward panning. To pan the image downward, the collaborative group pullsthe puck 210 against the top edge of the panning boundary 1902, thestronger the collaborative pull against the top barrier the faster thedownward panning. In the corners, the puck 210 can be used to pullagainst two panning boundaries 1902 at once, each proportional to thepull against the panning boundary 1902, thus allowing for diagonalpanning. In some embodiments, diagonal panning is enabled at all pointson the panning boundary 1902. To achieve this, we make the panningboundary 1902 ‘sticky’ so that the puck 210 does not slide along theedge of the panning boundary 1902. Thus, when the puck 210 touches thepanning boundary 1902 and is pulled into the panning boundary 1902 thepuck 210 stays stationary at the tangent location. The panning thenoccurs diagonally, in the inverse direction to the vector direction.When the puck 210 is pulled away from the panning boundary 1902, thepuck 210 releases (is no longer stationary at the tangent location).This enables diagonal panning in a simple and intuitive framework.

In some embodiments, the panning boundary 1902 is not visibly displayed.The users experience the puck 210 reaching the panning boundary 1902 bythe puck motion stopping (or slowing), and if the group continues tocollaboratively pull in that direction, the image 1102 pans in theopposite direction at a speed proportional to said collaborativepulling. In some embodiments, the puck 210 may be limited to a smallregion in the center of the image 1102, the panning boundary 1902 beingreached before the puck 210 reaches the edge of the actual displayedimage 1102. The panning boundary 1902 may be of a variety of shapes, forexample a circular region at the center of the image 1102. The panningboundary 1902 may be invisible, but still prevents the puck 210 fromleaving.

Referring next to FIG. 20, an exemplary map target area 2000 including acircular boundary is shown. Shown are the question display 302, thepointer 210, the target area 206, the image 1102, and the panningboundary 1902.

As previously described, the panning boundary 1902 need not becoincident with the extent of the image 1102. As shown in FIG. 20, thepanning boundary 1902 is a circular boundary surrounding the puck 210and implemented by the CCS 102 such that the puck 210 is unable to passout of the circular panning boundary 1902. When hitting up against thepanning boundary 1902 (which may or may not be visual), the users maycontinue to collectively pull the puck 210 in a direction that opposesthe panning boundary 1902 as indicated by their group intent vector. TheCCS 102 software will not allow the puck 210 to substantially pass thepanning boundary 1902, but the CIA and CCS 102 software are configuredto pan the image 1102 in a direction opposite to the group intentvector, with a speed proportional to the group intent vector.

In some embodiments wherein the puck 210 is bounded to a small region atthe center of the displayed image 1102, a virtual restoring force may beapplied by the CCS 102 software on the puck 210, said virtual restoringforce acting to return the puck 210 to the center of the image 1102 witha magnitude proportional to its distance from the center of the image1102. This can be represented as “virtual gravity” pulling the puck 210back to center. The panning is then implemented such that the directionof panning is opposite to the direction of the group intent vectorgenerated by the group of users as it pulls against gravity, the speedof the panning proportional to the magnitude of the group intent vectorgenerated as the collaborative group pulls against gravity. In this way,a virtual gravitational force can be used instead of a bounding line,edge, or surface.

In some such panning embodiments, the portions of the larger image thatare off screen and panned can be more than just a single image or a setof images, but a set of targets. For example, in one novel embodiment,the present invention is used to allow a collaborative intelligence tosentences by selecting a string of words, each of said words being on atarget field as a selectable target. This solves the problem that only asmall number of words can fit on the screen at once, limiting thevocabulary presented to users. Using the panning methods above, a largenumber of words can be included on a larger target area that extends faroff the screen, the target words being selectable under collaborativecontrol by first panning to that portion of the larger target area, thenby targeting the word with the pointer 210 (as disclosed previously).

For example, a full 850 words of the Ogden Basic English vocabularycould be presented on the larger target area, with a portion of thelarger target area shown as the target area 206, each of the words beinga discrete answer choice 208. A collaborative swarm of users would thenbe enabled to target any one of the words in the vocabulary by panningand targeting. In some embodiments, the words are arrangedalphabetically which allows users to know which way to pan to gettowards a desired word that is off screen. In other embodiments,collaborative panning can be combined with collaborative zooming suchthat a swarm of users can zoom out, quickly pan to a region, then zoomin to find a particular word.

While many embodiments are described herein, it is appreciated that thisinvention can have a range of variations that practice the same basicmethods and achieve the novel collaborative capabilities that have beendisclosed above. Many of the functional units described in thisspecification have been labeled as modules, in order to moreparticularly emphasize their implementation independence. For example, amodule may be implemented as a hardware circuit comprising custom VLSIcircuits or gate arrays, off-the-shelf semiconductors such as logicchips, transistors, or other discrete components. A module may also beimplemented in programmable hardware devices such as field programmablegate arrays, programmable array logic, programmable logic devices or thelike.

Modules may also be implemented in software for execution by varioustypes of processors. An identified module of executable code may, forinstance, comprise one or more physical or logical blocks of computerinstructions that may, for instance, be organized as an object,procedure, or function. Nevertheless, the executables of an identifiedmodule need not be physically located together, but may comprisedisparate instructions stored in different locations which, when joinedlogically together, comprise the module and achieve the stated purposefor the module.

Indeed, a module of executable code could be a single instruction, ormany instructions, and may even be distributed over several differentcode segments, among different programs, and across several memorydevices. Similarly, operational data may be identified and illustratedherein within modules, and may be embodied in any suitable form andorganized within any suitable type of data structure. The operationaldata may be collected as a single data set, or may be distributed overdifferent locations including over different storage devices, and mayexist, at least partially, merely as electronic signals on a system ornetwork.

While the invention herein disclosed has been described by means ofspecific embodiments, examples and applications thereof, numerousmodifications and variations could be made thereto by those skilled inthe art without departing from the scope of the invention set forth inthe claims.

What is claimed is:
 1. A collaborative intelligence system for enabling a plurality of users to jointly control a graphical pointer and collaboratively select a coordinate location within a graphical image during a collaborative session, the system comprising: a central collaboration server configured to run a collaboration server application and to exchange real-time data with each of a plurality of networked computing devices, each of the computing devices having a user interface and a display and running a collaborative intent application, wherein the collaborative intent application on each computing device is configured to: receive and display a graphical image on the display; display a collaboratively controlled graphical pointer over the graphical image; receive, repeatedly in real-time, from the central collaboration server, a pointer location; update, repeatedly in real-time, the displayed location of the collaboratively controlled graphical pointer such that it moves across the graphical image, the updated displayed location based at least in part on the received pointer location and the displayed location substantially synchronized among the plurality of computing devices; receive, repeatedly in real-time, user input through the user interface, the user input representing a user intent direction and a user intent magnitude, said user intent direction indicating a desired direction of motion of the collaboratively controlled graphical pointer with respect to the graphical image, said user intent magnitude indicating a desired speed of motion of the collaboratively controlled graphical pointer with respect to the graphical image; and send, repeatedly in real-time, a representation of the user intent direction and user intent magnitude to the central collaboration server; wherein the collaboration server application running on the central collaboration server is configured to: receive, repeatedly in real-time, the representation of the user intent direction and user intent magnitude, from each of the plurality of computing devices; determine, repeatedly in real-time, a group intent vector from the user intent directions and user intent magnitudes received from the plurality of computing devices, the group intent vector indicating a collaboratively desired motion of the collaboratively controlled graphical pointer; determine, repeatedly in real-time, the updated pointer location for the collaboratively controlled graphical pointer based at least in part upon the group intent vector; send, repeatedly in real-time, the pointer location to the plurality of computing devices; determine, repeatedly, whether a speed of the collaboratively controlled graphical pointer has fallen below a threshold value for more than a threshold amount of time; identify a target location upon the graphical image in response to the determination that the speed of the collaboratively controlled graphical pointer has fallen below the threshold value for more than the threshold amount of time; and send the target location to at least one of the plurality of computing devices.
 2. The collaborative intelligence system of claim 1, wherein the target location is a coordinate with respect to an origin location on the graphical image.
 3. The collaborative intelligence system of claim 1, wherein at least one of the plurality of computing devices displays a graphical identifier at the target location.
 4. The collaborative intelligence system of claim 1, wherein the graphical image is a geographic map.
 5. The collaborative intelligence system of claim 1, wherein the graphical image is a calendar.
 6. The collaborative intelligence system of claim 1, wherein during the collaborative selection process the graphical image is panned under collaborative control, said panning performed in response to the collaboratively controlled graphical pointer being collaboratively positioned so as to interact with a panning boundary.
 7. The collaborative intelligence system of claim 1, wherein user intent direction is indicated by a user controlling a second graphical pointer displayed on the display, the user intent direction being based upon an angle between the second graphical pointer and the collaboratively controlled graphical pointer.
 8. The collaborative intelligence system of claim 1, wherein user intent magnitude is indicated by a user by controlling a second graphical pointer displayed on the display, the user intent magnitude being based upon a distance between the second graphical pointer and the collaboratively controlled graphical pointer.
 9. The collaborative intelligence system of claim 1, wherein the speed threshold increases over time during the collaborative session.
 10. A collaborative intelligence system for enabling a plurality of users to jointly control panning of a display of a graphical image, the system comprising: a central collaboration server configured to run a collaboration server application and to exchange real-time data with each of a plurality of networked computing devices, each of the computing devices having a user interface and a display and running a collaborative intent application, wherein the collaborative intent application on each computing device is configured to: receive and display a portion of a graphical image in a display area of the display, wherein only a portion of the graphical image is visible as a result of the graphical image being larger than the display area; display a collaboratively controlled graphical pointer over the displayed portion of the graphical image; receive, repeatedly in real-time, from the collaborative intelligence server, a pointer location; update, repeatedly in real-time, the displayed location of the collaboratively controlled graphical pointer such that it moves with respect to the display area, the updated location based at least in part on the received pointer location and displayed location substantially synchronized among the plurality of computing devices; receive, repeatedly in real-time, user input through the user interface, the user input representing a user intent direction and a user intent magnitude, said user intent direction indicating a desired direction of motion of the collaboratively controlled graphical pointer, said user intent magnitude indicating a desired speed of motion of the collaboratively controlled graphical pointer; and send, repeatedly in real-time, a representation of the user intent direction and user intent magnitude to the collaborative intelligence server; wherein the collaboration server application running on the central collaboration server is configured to: receive, repeatedly in real-time, the representation of the user intent direction and user intent magnitude, from each of the plurality of computing devices; determine, repeatedly in real-time, a group intent vector from the user intent directions and user intent magnitudes received from the plurality of computing devices, the group intent vector indicating a collaboratively desired motion of the collaboratively controlled graphical pointer determine, repeatedly in real-time, an updated pointer location for the collaboratively controlled graphical pointer based at least in part upon the group intent vector; send, repeatedly in real-time, the pointer location to the plurality of computing devices; determine, repeatedly, whether the collaboratively controlled graphical pointer has reached a panning boundary region defined with respect to the display area, and if the graphical pointer has reached the panning boundary region, prevent the pointer from leaving the panning boundary region; and coordinate the panning of the graphical image, a direction of the panning being substantially opposite to the direction of the group intent vector when the collaboratively controlled pointer has reached the panning boundary region.
 11. The collaborative intelligence system of claim 10 wherein a speed of the panning of the graphical image is proportional to a magnitude of the group intent vector.
 12. The collaborative intelligence system of claim 10 wherein the panning boundary region is a small region near the center of the display area.
 13. The collaborative intelligence system of claim 10 wherein the collaboratively controlled graphical pointer is pulled back to the center of the panning boundary region by a simulated gravitational force.
 14. The collaborative intelligence system of claim 10 wherein the graphical image is a geographic map.
 15. The collaborative intelligence system of claim 10 wherein the graphical image includes a plurality of distinct graphical targets.
 16. The collaborative intelligence system of claim 10 wherein user intent direction is indicated by a user by controlling a second graphical pointer, the user intent direction being based upon an angle between the second graphical pointer and the collaboratively controlled graphical pointer.
 17. The collaborative intelligence system of claim 10 wherein user intent magnitude is indicated by a user by controlling a second graphical pointer displayed on the display area, the user intent magnitude being based upon a distance between the second graphical pointer and the collaboratively controlled graphical pointer.
 18. The collaborative intelligence system of claim 10 wherein the collaborative intent application is further configured to determine, repeatedly, whether a speed of the collaboratively controlled graphical pointer has fallen below a threshold value for more than a threshold amount of time and if so, trigger a zoom feature wherein the graphical image is zoomed in on the display of the plurality of computing devices.
 19. The collaborative intelligence system of claim 10 wherein the collaborative intent application is further configured to determine, repeatedly, whether a speed of the collaboratively controlled graphical pointer has fallen below a threshold value for more than a threshold amount of time and in response identify a target location upon the graphical image in response to the determination, the target location being sent to at least one of the plurality of computing devices. 